dotnet/BenchmarkDotNet

Provide support for running using an existing .NET exe

开放

#568 创建于 2017年10月17日

 (1 条评论) (0 个反应) (0 位负责人)C# (1,063 个派生)batch import
help wantedup-for-grabs

仓库指标

星标
 (11,480 个星标)
PR 合并指标
 (PR 指标待抓取)

描述

So, here's my scenario:

I would like to use Benchmark.NET to measure the performance of the Roslyn compiler end-to-end, meaning my benchmark is basically csc -noconfig @repro.rsp.

I'll grant you, this isn't an obvious fit for BenchmarkDotNet, but technically I'm still benchmarking .NET code and there are a lot of things, like providing easy swapping of runtimes, that BenchmarkDotNet is great for. Perhaps most importantly, I don't really want to redo all the statistical analysis that's already built into BenchmarkDotNet.

So anyway, the problem is that I essentially need to do a process launch. The normal operation of BenchmarkDotNet would do two process launches, but I've managed to kind of get this working using the InProcess running mode (and manually working around the default timeout -- why is there a timeout on the InProcess mode??), but this is really hacked up.

Here's my proposal:

We should add support for the BenchmarkAttribute on properties and fields of type ProcessStartInfo. The contract here is that ProcessStartInfo would contain an arguments array that includes as the first argument the path to the managed EXE. BenchmarkDotNet would then plug in a process executor that would slot in the appropriate runtime and runtime options and then execute the given StartInfo. Other things would work somewhat analogously.

This would allow me to keep all my code in the external EXE, which is much easier to compile standalone, and also easily allow me to swap out runtimes.

贡献者指南