Incorrect stack traceback when try except finally is used in jupyter notebooks with pandas

Reproduce the issue in a Jupyter notebook using the provided pandas DataFrame example and compare it with the dictionary KeyError example. Inspect how ipykernel handles and renders traceback frames for try/except/finally blocks; done means the traceback points to the DataFrame construction rather than the finally block in both Jupyter and VSCode notebooks.

Originally opened by @dvorst in https://github.com/jupyter/notebook/issues/7242

[✓] I checked the documentation and found no answer [✓] I checked to make sure that this issue has not already been filed [✓] I'm reporting the issue to the correct repository

Description

The stack traceback of some errors is displayed incorrectly. Below are two examples of an incorrect stacktrace and correct stacktrace. The incorrect one creates a pandas dataframe in a faulty manner, raising an exception, its stacktrace points to a command in the finally block instead of the pandas dataframe creation which is the actual issue.

The correct example shows how a non-existing key of a dictionary is called, its stacktrace does not point to a statement in the finally block but rather points to the line that actually causes the issue.

Some things i've tried or noticed:

* the correct example contains hyperlinks to each code-line, while the incorrect one doesn't, maybe this is a hint to the cause. I therefore included the stacktrace of both examples in raw text as well as html.

* running the'incorrect stacktrace' example in pure python (.py file) results in a **correct** stacktrace

* this issue occurs when hosting jupyter notebooks locally, but also in VSCode notebooks using the ipykernel

* Thinking that C-code extensions of Python could be a potential source, I also tried raising errors using torch (torch.tensor('invalid_type') but this results in a correct stacktrace.

* aside of torch i also tried incorrectly initiating classes with other packages, this also lead to a correct stacktrace

incorrect stacktrace

code

import pandas as pd

try:
    pd.DataFrame(1)
except:
    print("except")
    raise
finally:
    print("finally")

stacktrace text

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/var/folders/j4/hhybm8xx3pb7nnmwfptdqng00000gp/T/ipykernel_40674/1767429816.py in ?()
      5 except:
      6     print("except")
      7     raise
      8 finally:
----> 9     print("finally")

~/Documents/projects/notebook-stacktrace-bug/.venv/lib/python3.12/site-packages/pandas/core/frame.py in ?(self, data, index, columns, dtype, copy)
    871                 )
    872         # For data is scalar
    873         else:
    874             if index is None or columns is None:
--> 875                 raise ValueError("DataFrame constructor not properly called!")
    876 
    877             index = ensure_index(index)
    878             columns = ensure_index(columns)

ValueError: DataFrame constructor not properly called!

stacktrace html

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[/var/folders/j4/hhybm8xx3pb7nnmwfptdqng00000gp/T/ipykernel_40674/1767429816.py](https://file+.vscode-resource.vscode-cdn.net/var/folders/j4/hhybm8xx3pb7nnmwfptdqng00000gp/T/ipykernel_40674/1767429816.py) in ?()
      5 except:
      6     print("except")
      7     raise
      8 finally:
----> 9     print("finally")

[~/Documents/projects/notebook-stacktrace-bug/.venv/lib/python3.12/site-packages/pandas/core/frame.py](https://file+.vscode-resource.vscode-cdn.net/Users/dvorst/Documents/projects/notebook-stacktrace-bug/~/Documents/projects/notebook-stacktrace-bug/.venv/lib/python3.12/site-packages/pandas/core/frame.py) in ?(self, data, index, columns, dtype, copy)
    871                 )
    872         # For data is scalar
    873         else:
    874             if index is None or columns is None:
--> 875                 raise ValueError("DataFrame constructor not properly called!")
    876 
    877             index = ensure_index(index)
    878             columns = ensure_index(columns)

ValueError: DataFrame constructor not properly called!

correct stacktrace

code

some_dict = {}
try:
    some_dict['non-existent-key']
except:
    print("except")
    raise
finally:
    print("finally")

stacktrace text

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[6], line 3
      1 some_dict = {}
      2 try:
----> 3     some_dict['non-existent-key']
      4 except:
      5     print("except")

KeyError: 'non-existent-key'

stacktrace html

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[6], [line 3](vscode-notebook-cell:?execution_count=6&line=3)
      [1](vscode-notebook-cell:?execution_count=6&line=1) some_dict = {}
      [2](vscode-notebook-cell:?execution_count=6&line=2) try:
----> [3](vscode-notebook-cell:?execution_count=6&line=3)     some_dict['non-existent-key']
      [4](vscode-notebook-cell:?execution_count=6&line=4) except:
      [5](vscode-notebook-cell:?execution_count=6&line=5)     print("except")

KeyError: 'non-existent-key'

Context

This occurs in the browser with notebook 7.0.7, but also in VSCode notebooks with ipykernel 6.29.2

* Operating System and version: macOS 14.2

* Browser and version: Firefox 122.0.1 though this shouldn't matter as the issue also occurs in VSCode

IPython : 8.21.0 ipykernel : 6.29.2 ipywidgets : not installed jupyter_client : 8.6.0 jupyter_core : 5.7.1 jupyter_server : 2.12.5 jupyterlab : 4.1.0 nbclient : 0.9.0 nbconvert : 7.16.0 nbformat : 5.9.2 notebook : 7.0.7 qtconsole : not installed traitlets : 5.14.1

pandas : 2.2.0 vscode. : 1.86.0
Troubleshoot Output

I did not include the command line output as this caused the maximum amount of characters allowed to be exceeded.