Hedgehog with convenience attributes for xUnit.net.
- Test method arguments generated with a custom
GenX.auto... - ...or with a custom Generator.
Property.checkcalled for each test.
This readme is for C#. Go here for F# documentation.
Install the Hedgehog.Xunit package from Visual Studio's Package Manager Console:
PM> Install-Package Hedgehog.XunitSuppose you have a test that uses Hedgehog.Experimental and looks similar to the following:
using Hedgehog;
using Hedgehog.Linq;
using Hedgehog.Xunit;
using Property = Hedgehog.Linq.Property;
public class DocumentationSamples
{
[Fact]
public void Reversing_a_list_twice_yields_the_original_list()
{
var gen = GenX.auto<List<int>>();
var prop = from xs in Property.ForAll(gen)
let testList = Enumerable.Reverse(xs).Reverse().ToList()
select Assert.Equal(xs, testList);
prop.Check();
}
}Then using Hedgehog.Xunit, you can simplify the above test to
[Property]
public void Reversing_a_list_twice_yields_the_original_list_with_xunit(List<int> xs)
{
var testList = Enumerable.Reverse(xs).Reverse().ToList();
Assert.Equal(xs, testList);
}Hedgehog.Xunit provides the following attributes:
[Property]Extends xUnit'sFactto call Hedgehog'sproperty.[Properties]Configures allPropertytagged tests in a module or class.GenAttributeExtend this attribute to set a parameter's generator.[Recheck]Run a test with a specificSizeandSeed.
👉 All code in this document is available here.
Methods with [Property] have their arguments generated by GenX.auto.
using global::Xunit.Abstractions;
public class DocumentationSamples
{
private readonly ITestOutputHelper _output;
public DocumentationSamples(ITestOutputHelper output)
{
_output = output;
}
[Property]
public void Can_generate_an_int(
int i)
{
_output.WriteLine($"Test input: {i}");
}
}Test input: 0
Test input: -1
Test input: 1
...
Test input: 522317518
Test input: 404306656
Test input: 1550509078
Property.check is called.
[Property]
public bool Will_fail(bool value) => value;System.Exception: *** Failed! Falsifiable (after 5 tests):
[false]
Hedgehog.Xunit.TestReturnedFalseException: Test returned `false`.
If the test returns FSharp.Control.Async<T> or Task<T>, then Async.RunSynchronously is called, which blocks the thread. This may have significant performance implications as tests run 100 times by default.
[Property]
public async Task Task_with_exception_shrinks(int i)
{
await Task.Delay(100);
if (i > 10) throw new Exception();
}System.Exception: *** Failed! Falsifiable (after 14 tests and 2 shrinks):
[11]
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
---> System.AggregateException: One or more errors occurred. (Exception of type 'System.Exception' was thrown.)
---> System.Exception: Exception of type 'System.Exception' was thrown.
A test returning an FSharp.Core.Result in an Error state will be treated as a failure.
[Property]
public FSharpResult<int, string> Result_with_Error_shrinks(int i) =>
i < 10
? FSharpResult<int, string>.NewOk(i)
: FSharpResult<int, string>.NewError("humbug!");System.Exception: *** Failed! Falsifiable (after 15 tests and 1 shrink):
[10]
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
---> System.Exception: Result is in the Error case with the following value:
"humbug!"
Tests returning Async<Result<T, TError>> or Task<Result<T, TError>> are run synchronously and are expected to be in the Ok state.
Tests returning a Property<unit> or Property<bool> will have Property.check automatically called:
[Property]
public Property<bool> Returning_a_failing_property_bool_with_an_external_number_gen_fails_and_shrinks(int i) =>
from fifty in Property.ForAll(Hedgehog.Gen.constant(50))
select i <= fifty;System.Exception: *** Failed! Falsifiable (after 25 tests and 4 shrinks):
[51]
50
[Property]'s constructor may take several arguments:
AutoGenConfigandAutoGenConfigArgs: Set anAutoGenConfigto use when generating arguments.Tests: Specifies the number of tests to be run.Shrinks: Specifies the maximal number of shrinks that may run.Size: Sets theSizeto a value for all runs.
The Property attribute extends Xunit.FactAttribute, so it may also take DisplayName, Skip, and Timeout.
GenX.defaults is the AutoGenConfig used by default.
Here's how to add your own generators:
- Create a class with a single static property or method that returns an instance of
AutoGenConfig. - Provide the type of this class as an argument to
[Property]. (This works around the constraint thatAttributeparameters must be a constant.)
public class AutoGenConfigContainer
{
public static AutoGenConfig _ =>
GenX.defaults.WithGenerator(Gen.Int32(Range.FromValue(13)));
}
[Property(typeof(AutoGenConfigContainer))]
public bool This_test_passes_because_always_13(int i) => i == 13;If the method takes arguments, you must provide them using AutoGenConfigArgs.
public class ConfigWithArgs
{
public static AutoGenConfig _(
string word,
int number) =>
GenX.defaults
.WithGenerator(Hedgehog.Gen.constant(word))
.WithGenerator(Hedgehog.Gen.constant(number));
}
[Property(AutoGenConfig = typeof(ConfigWithArgs), AutoGenConfigArgs = new object[] { "foo", 13 })]
public bool This_also_passes(string s, int i) =>
s == "foo" && i == 13;Specifies the number of tests to be run, though more or less may occur due to shrinking or early failure.
[Property(tests: 3)]
public void This_runs_3_times() => _output.WriteLine($"Test run");Specifies the maximal number of shrinks that may run.
[Property(Shrinks = 0]
public void No_shrinks_occur(int i)
{
if (i > 50)
{
throw new Exception("oops");
}
}Sets the Size to a value for all runs.
[Property(Size = 2)]
public void i_mostly_ranges_between_neg_1_and_1(int i) => _output.WriteLine(i.ToString());This optional attribute can decorate modules or classes. It sets default arguments for AutoGenConfig, AutoGenConfigArgs, Tests, Shrinks, and Size. These will be overridden by any explicit arguments on [Property].
public class Int13
{
public static AutoGenConfig _ => GenX.defaults.WithGenerator(Hedgehog.Gen.constant(13));
}
public class Int2718
{
public static AutoGenConfig _ => GenX.defaults.WithGenerator(Hedgehog.Gen.constant(2718));
}
[Properties(typeof(Int13), 1)]
public class PropertiesSample
{
[Property]
public bool this_passes_and_runs_once(
int i) =>
i == 13;
[Property(typeof(Int2718), 2)]
public bool this_passes_passes_and_runs_twice(
int i) =>
i == 2718;
}To assign a generator to a test's parameter, extend GenAttribute and override Generator:
public class Five : GenAttribute<int>
{
public override Gen<int> Generator => Gen.Int32(Range.FromValue(5));
}
[Property]
public bool Can_set_parameter_as_5(
[Five] int five) =>
five == 5;Here's a more complex example of GenAttribute that takes a parameter and overrides Property's AutoGenConfig:
public class AutoGenConfigContainer
{
public static AutoGenConfig _ =>
GenX.defaults.WithGenerator(Gen.Int32(Range.FromValue(13)));
}
public class ConstInt : GenAttribute<int>
{
private readonly int _i;
public ConstInt(int i)
{
_i = i;
}
public override Gen<int> Generator => Gen.Int32(Range.FromValue(_i));
}
[Property(typeof(AutoGenConfigContainer))]
public bool GenAttribute_overrides_Property_AutoGenConfig(int thirteen, [ConstInt(6)] int six) =>
thirteen == 13 && six == 6;,This optional method attribute invokes Property.recheck with the given Size and Seed. It must be used with Property.
[Property]
[Recheck("44_13097736474433561873_6153509253234735533_")]
public bool this_passes(int i) => i == 12345;Use named arguments to select the desired constructor overload.
[Properties(Tests = 13, AutoGenConfig = typeof(AutoGenConfigContainer))]
public class __
{
[Property(AutoGenConfig = typeof(AutoGenConfigContainer), Tests = 2718, Skip = "just because")]
public void Not_sure_why_youd_do_this_but_okay()
{
}
}Consider extending PropertyAttribute or PropertiesAttribute to hardcode commonly used arguments. It also works with GenAttribute.
public class Five : GenAttribute<int>
{
public override Gen<int> Generator => Gen.Int32(Range.FromValue(5));
}
public class Int13
{
public static AutoGenConfig _ =>
GenX.defaults.WithGenerator(Gen.Int32(Range.FromValue(13)));
}
public class PropertyInt13Attribute : PropertyAttribute
{
public PropertyInt13Attribute() : base(typeof(Int13)) { }
}
[PropertyInt13]
public bool This_passes(int thirteen, [Five] int five) => thirteen == 13 && five == 5;
public class PropertiesInt13Attribute : PropertiesAttribute
{
public PropertiesInt13Attribute() : base(typeof(Int13)) { }
}
[PropertiesInt13]
public class ___
{
[Property]
public bool This_also_passes(int thirteen, [Five] int five) => thirteen == 13 && five == 5;
}