dotnet/runtime

Safer failure with COM/C++/vtable interop and RTTI

Open

#86,458 opened on 2023年5月18日

GitHub で見る
 (7 comments) (0 reactions) (0 assignees)C# (5,445 forks)batch import
area-Interop-coreclrhelp wanted

Repository metrics

Stars
 (17,886 stars)
PR merge metrics
 (平均マージ 12d 11h) (30d で 661 merged PRs)

説明

Interop with COM/vtables is currently performed by allocating an array of function pointers. However, this may be dangerous if some C++ code attempts to use dynamic_cast, typeid, or some other feature that requires the existence of RTTI. This is because the RTTI pointer is located just before the first entry of the vtable, which is out of bounds of the memory that was allocated.

To my knowledge, both the Microsoft and Itanium ABIs will treat a nullptr value for the RTTI pointer to mean "no RTTI is present for this type" (in the case of MSVC, __RTDynamicCast will catch an AV and treat that as "no RTTI"). I am not sure how the older GCC ABI deals with this.

Would it be desirable for the COM and vtable source generators (and the general guidance on vtable interop in .NET) to be changed from this:

void** vtable = new void*[methodCount];
// fill in vtable

to this?

void** vtable = new void*[methodCount + 1];
vtable[0] = nullptr;
vtable++;
// fill in vtable

It would ensure that the RTTI pointer is null and the proper failure messages can be displayed instead of potentially reading an invalid (but readable) pointer.

I think supporting RTTI is out of scope for .NET interop, but adding a dummy null pointer before the vtable seems like a good "best effort" to ensure that the failure is still deterministic if some code attempts to use RTTI with an interop vtable.

コントリビューターガイド