is_async=True no longer works on Jupyter

[astrojuanlu]

Description

As per title

Steps to Reproduce

1. Follow the QA notes of Refactor _run_node_async to use asyncio #4562 (comment) with Kedro 0.19.12, run session.run("adata_processing", runner=SequentialRunner(is_async=True)) on Jupyter, everything works ✔
2. Install main branch
3. Run session.run("adata_processing", runner=SequentialRunner(is_async=True)) on Jupyter, it breaks 💥

Expected Result

Continues to work as it used to

Actual Result

[03/25/25 17:53:41] INFO     Kedro project kedro-async-test                                          [session.py](file:///Users/juan_cano/Projects/QuantumBlackLabs/tmp/kedro-async-test/.venv/lib/python3.13/site-packages/kedro/framework/session/session.py):[329](file:///Users/juan_cano/Projects/QuantumBlackLabs/tmp/kedro-async-test/.venv/lib/python3.13/site-packages/kedro/framework/session/session.py#329)

                    WARNING  /Users/juan_cano/Projects/QuantumBlackLabs/tmp/kedro-async-test/.venv/ [warnings.py](file:///Users/juan_cano/.local/share/uv/python/cpython-3.13.0-macos-aarch64-none/lib/python3.13/warnings.py):[110](file:///Users/juan_cano/.local/share/uv/python/cpython-3.13.0-macos-aarch64-none/lib/python3.13/warnings.py#110)
                             lib/python3.13/site-packages/kedro/io/data_catalog.py:165:                            
                             KedroDeprecationWarning: `DataCatalog` has been deprecated and will be                
                             replaced by `KedroDataCatalog`, in Kedro 1.0.0.Currently some                         
                             `KedroDataCatalog` APIs have been retained for compatibility with                     
                             `DataCatalog`, including the `datasets` property and the                              
                             `get_datasets`, `_get_datasets`, `add`,` list`, `add_feed_dict`, and                  
                             `shallow_copy` methods. These will be removed or replaced with updated                
                             alternatives in Kedro 1.0.0. For more details, refer to the                           
                             documentation:                                                                        
                             https://docs.kedro.org/en/stable/data/index.html#kedrodatacatalog-expe                
                             rimental-feature                                                                      
                               warnings.warn(                                                                      
                                                                                                                   

                    INFO     Kedro is sending anonymous usage data with the sole purpose of improving [plugin.py](file:///Users/juan_cano/Projects/QuantumBlackLabs/tmp/kedro-async-test/.venv/lib/python3.13/site-packages/kedro_telemetry/plugin.py):[233](file:///Users/juan_cano/Projects/QuantumBlackLabs/tmp/kedro-async-test/.venv/lib/python3.13/site-packages/kedro_telemetry/plugin.py#233)
                             the product. No personal data or IP addresses are stored on our side. If              
                             you want to opt out, set the `KEDRO_DISABLE_TELEMETRY` or `DO_NOT_TRACK`              
                             environment variables, or create a `.telemetry` file in the current                   
                             working directory with the contents `consent: false`. Read more at                    
                             https://docs.kedro.org/en/stable/configuration/telemetry.html                         

[03/25/25 17:53:42] INFO     Asynchronous mode is enabled for loading and saving data                 [runner.py](file:///Users/juan_cano/Projects/QuantumBlackLabs/tmp/kedro-async-test/.venv/lib/python3.13/site-packages/kedro/runner/runner.py):[125](file:///Users/juan_cano/Projects/QuantumBlackLabs/tmp/kedro-async-test/.venv/lib/python3.13/site-packages/kedro/runner/runner.py#125)

                    WARNING  No nodes ran. Repeat the previous command to attempt a new run.          [runner.py](file:///Users/juan_cano/Projects/QuantumBlackLabs/tmp/kedro-async-test/.venv/lib/python3.13/site-packages/kedro/runner/runner.py):[334](file:///Users/juan_cano/Projects/QuantumBlackLabs/tmp/kedro-async-test/.venv/lib/python3.13/site-packages/kedro/runner/runner.py#334)

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ in <module>:1                                                                                    │
│                                                                                                  │
│ ❱ 1 session.run("adata_processing", runner=SequentialRunner(is_async=True))                      │
│   2                                                                                              │
│                                                                                                  │
│ /Users/juan_cano/Projects/QuantumBlackLabs/tmp/kedro-async-test/.venv/lib/python3.13/site-packag │
│ es/kedro/framework/session/session.py:399 in run                                                 │
│                                                                                                  │
│   396 │   │   │   run_params=record_data, pipeline=filtered_pipeline, catalog=catalog            │
│   397 │   │   )                                                                                  │
│   398 │   │   try:                                                                               │
│ ❱ 399 │   │   │   run_result = runner.run(                                                       │
│   400 │   │   │   │   filtered_pipeline, catalog, hook_manager, session_id                       │
│   401 │   │   │   )                                                                              │
│   402 │   │   │   self._run_called = True                                                        │
│                                                                                                  │
│ /Users/juan_cano/Projects/QuantumBlackLabs/tmp/kedro-async-test/.venv/lib/python3.13/site-packag │
│ es/kedro/runner/runner.py:129 in run                                                             │
│                                                                                                  │
│   126 │   │   │   │   "Asynchronous mode is enabled for loading and saving data"                 │
│   127 │   │   │   )                                                                              │
│   128 │   │                                                                                      │
│ ❱ 129 │   │   self._run(pipeline, catalog, hook_or_null_manager, session_id)  # type: ignore[a   │
│   130 │   │                                                                                      │
│   131 │   │   self._logger.info("Pipeline execution completed successfully.")                    │
│   132                                                                                            │
│                                                                                                  │
│ /Users/juan_cano/Projects/QuantumBlackLabs/tmp/kedro-async-test/.venv/lib/python3.13/site-packag │
│ es/kedro/runner/sequential_runner.py:72 in _run                                                  │
│                                                                                                  │
│   69 │   │   │   │   "Using synchronous mode for loading and saving data. Use the --async fla    │
│   70 │   │   │   │   "for potential performance gains. https://docs.kedro.org/en/stable/nodes    │
│   71 │   │   │   )                                                                               │
│ ❱ 72 │   │   super()._run(                                                                       │
│   73 │   │   │   pipeline=pipeline,                                                              │
│   74 │   │   │   catalog=catalog,                                                                │
│   75 │   │   │   hook_manager=hook_manager,                                                      │
│                                                                                                  │
│ /Users/juan_cano/Projects/QuantumBlackLabs/tmp/kedro-async-test/.venv/lib/python3.13/site-packag │
│ es/kedro/runner/runner.py:239 in _run                                                            │
│                                                                                                  │
│   236 │   │   │   │   │   │   hook_manager=hook_manager,                                         │
│   237 │   │   │   │   │   │   is_async=self._is_async,                                           │
│   238 │   │   │   │   │   │   session_id=session_id,                                             │
│ ❱ 239 │   │   │   │   │   ).execute()                                                            │
│   240 │   │   │   │   │   done_nodes.add(node)                                                   │
│   241 │   │   │   │   except Exception:                                                          │
│   242 │   │   │   │   │   self._suggest_resume_scenario(pipeline, done_nodes, catalog)           │
│                                                                                                  │
│ /Users/juan_cano/Projects/QuantumBlackLabs/tmp/kedro-async-test/.venv/lib/python3.13/site-packag │
│ es/kedro/runner/task.py:80 in execute                                                            │
│                                                                                                  │
│    77 │   │   if self.is_async:                                                                  │
│    78 │   │   │   import asyncio                                                                 │
│    79 │   │   │                                                                                  │
│ ❱  80 │   │   │   node = asyncio.run(                                                            │
│    81 │   │   │   │   self._run_node_async(                                                      │
│    82 │   │   │   │   │   self.node,                                                             │
│    83 │   │   │   │   │   self.catalog,                                                          │
│                                                                                                  │
│ /Users/juan_cano/.local/share/uv/python/cpython-3.13.0-macos-aarch64-none/lib/python3.13/asyncio │
│ /runners.py:190 in run                                                                           │
│                                                                                                  │
│   187 │   """                                                                                    │
│   188 │   if events._get_running_loop() is not None:                                             │
│   189 │   │   # fail fast with short traceback                                                   │
│ ❱ 190 │   │   raise RuntimeError(                                                                │
│   191 │   │   │   "asyncio.run() cannot be called from a running event loop")                    │
│   192 │                                                                                          │
│   193 │   with Runner(debug=debug, loop_factory=loop_factory) as runner:                         │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
RuntimeError: asyncio.run() cannot be called from a running event loop

Your Environment

• Kedro version used (pip show kedro or kedro -V):
• Python version used (python -V): tested with 3.13
• Operating system and version: macOS

[ravi-kumar-pilla]

I am able to reproduce this issue. Thanks for raising @astrojuanlu

[ravi-kumar-pilla]

Hi @astrojuanlu ,

I tried a fix suggested by some in this discussion. Tested it and it is working, but there are also comments around not using nested event loops. So not really sure on other possibilities. If we have anyone with prior experience on nested event loops, please let me know

cc: @noklam @ElenaKhaustova

Thank you