[BUG]: 'terminate called after throwing an instance of' in `def_buffer` method

Required prerequisites

• Make sure you've read the documentation. Your issue may be addressed there.
• Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
• Consider asking first in the Gitter chat room or in a Discussion.

Problem description

It seems that if I throw an exception from a def_buffer method, this is not translated properly to a Python exception and instead terminates the program with something like:

terminate called after throwing an instance of 'std::invalid_argument'
  what():  unimplemented array format.
Aborted

Reproducible example code

I believe the following minimal repro causes this but I have not yet built it separately/verified outside of my project.

struct Foobar;
py::class_<Foobar>(m, "Foobar")
  .def(py::init<>())
  .def_buffer([](Foobar &) {
    throw std::invalid_argument("boom");
  });

import numpy as np
import my_module
np.array(my_module.Foobar())