dotnet/BenchmarkDotNet

Provide support for running using an existing .NET exe

Open

#568 创建于 2017年10月17日

在 GitHub 查看
 (1 评论) (0 反应) (0 负责人)C# (11,415 star) (1,054 fork)batch import
help wantedup-for-grabs

描述

So, here's my scenario:

I would like to use Benchmark.NET to measure the performance of the Roslyn compiler end-to-end, meaning my benchmark is basically csc -noconfig @repro.rsp.

I'll grant you, this isn't an obvious fit for BenchmarkDotNet, but technically I'm still benchmarking .NET code and there are a lot of things, like providing easy swapping of runtimes, that BenchmarkDotNet is great for. Perhaps most importantly, I don't really want to redo all the statistical analysis that's already built into BenchmarkDotNet.

So anyway, the problem is that I essentially need to do a process launch. The normal operation of BenchmarkDotNet would do two process launches, but I've managed to kind of get this working using the InProcess running mode (and manually working around the default timeout -- why is there a timeout on the InProcess mode??), but this is really hacked up.

Here's my proposal:

We should add support for the BenchmarkAttribute on properties and fields of type ProcessStartInfo. The contract here is that ProcessStartInfo would contain an arguments array that includes as the first argument the path to the managed EXE. BenchmarkDotNet would then plug in a process executor that would slot in the appropriate runtime and runtime options and then execute the given StartInfo. Other things would work somewhat analogously.

This would allow me to keep all my code in the external EXE, which is much easier to compile standalone, and also easily allow me to swap out runtimes.

贡献者指南

Provide support for running using an existing .NET exe · dotnet/BenchmarkDotNet#568 | Good First Issue