diff --git a/.agents/skills/verify-querykit/features/README.md b/.agents/skills/verify-querykit/features/README.md index d0a8fdb..2729541 100644 --- a/.agents/skills/verify-querykit/features/README.md +++ b/.agents/skills/verify-querykit/features/README.md @@ -21,6 +21,8 @@ The driver model is `Recipe`. Every recipe has one `Author` and zero or more `In | Salt Bread | Julia Child | 4 | 3.25 | true | Private (2) | 2024-06-10 12:00 | 1950-05-20 | 003 | Sliced | 12:15:30.25 | Knead and bake | bread, Winner | salt:5, flour:3 | | Plain Water | Anonymous | 1 | 0.00 | true | Private (2) | 2023-12-31 23:59 | null | 004 | null | null | null | (none) | (none) | +`CreatedAt` is a `timestamp with time zone` column. `LocalCreatedAt` holds the same wall-clock time as `CreatedAt` in a `timestamp without time zone` column. + ## Driving conventions - Start every recipe from the baseline. The driver only reads data, so no recipe changes the seed rows. diff --git a/.agents/skills/verify-querykit/features/configuration.md b/.agents/skills/verify-querykit/features/configuration.md index 2a014e8..0d71b7a 100644 --- a/.agents/skills/verify-querykit/features/configuration.md +++ b/.agents/skills/verify-querykit/features/configuration.md @@ -14,6 +14,7 @@ A developer passes a `QueryKitConfiguration` to change how QueryKit reads the in - `config-max-depth` limits the depth of dotted paths with `MaxPropertyDepth`. - `config-upper` uses `ToUpper()` for case-insensitive operators with `CaseInsensitiveComparison = CaseInsensitiveMode.Upper`. - `config-parameterized` sends filter values as SQL parameters with `ParameterizeFilterValues = true`. +- `config-datetime-kind` reads a `DateTime` value without an offset as `Unspecified` with `DateTimeKindForValuesWithoutOffset`. - `config-remove-ignored` drops an ignored clause instead of replacing it with `True == True`, with `IgnoredClauseBehavior = IgnoredClauseBehavior.Remove`. - `config-small-limits` lowers the parse limits with `MaxInputLength` and `MaxNestingDepth`. - `config-nesting-depth` counts each parenthesized group against `MaxNestingDepth`. A `(` or `)` inside a quoted value does not change the count. @@ -29,7 +30,7 @@ Preconditions: - A run is up and `qk doctor` prints only `ok` lines. - The seed data matches `features/README.md`. -- `qk configs` lists `aliases`, `loose-names`, `derived`, `custom-operation`, `word-operators`, `hidden-price`, `allow-unknown`, `max-depth-0`, `upper`, `parameterized`, `remove-ignored`, `small-limits`, and `depth-10`. +- `qk configs` lists `aliases`, `loose-names`, `derived`, `custom-operation`, `word-operators`, `hidden-price`, `allow-unknown`, `max-depth-0`, `upper`, `parameterized`, `unspecified-datetime`, `unspecified-datetime-parameterized`, `remove-ignored`, `small-limits`, and `depth-10`. - **Query names.** Run `qk run configuration-query-name --config aliases --filter 'chef == "Julia Child" && name _= "S"'`. Both targets give `["Salt Bread"]`. - **Query names that are not identifiers.** Run `qk run configuration-loose-query-names --config loose-names --filter 'recipe-title == "Pancakes" || _stars > 4 || chef name == "Gordon Ramsay"'`. Both targets give `["Pancakes", "Beef Stew"]`. @@ -42,6 +43,8 @@ Preconditions: - **Max depth.** Run `qk run configuration-max-depth --config max-depth-0 --filter 'Author.Name == "Julia Child"'`. Exit `2`. Both targets have `error.type` `QueryKit.Exceptions.QueryKitPropertyDepthExceededException`. - **Upper mode.** Run `qk run configuration-upper --config upper --filter 'Title @=* "bread"'`. Both targets give `["Salt Bread"]`. The `sql` contains `upper(r."Title")`. - **Parameterized values.** Run `qk run configuration-parameterized --config parameterized --filter 'Title == "Pancakes"'`. Both targets give `["Pancakes"]`. The `sql` contains a `@` parameter instead of the literal `'Pancakes'`. +- **Date without an offset, default.** Run `qk run configuration-datetime-utc --filter 'CreatedAt ^^ [2024-01-15T08:00:00]'`. Both targets give `["Pancakes"]`. The same filter on `LocalCreatedAt` fails on Postgres with an Npgsql `ArgumentException`, because the value is `Utc`. +- **Date without an offset, unspecified.** Run `qk run configuration-datetime-unspecified --config unspecified-datetime --filter 'LocalCreatedAt == "2024-01-15T08:00:00"'`. Both targets give `["Pancakes"]`. The `sql` contains `TIMESTAMP '2024-01-15T08:00:00'`. Repeat with `--config unspecified-datetime-parameterized` and with `'LocalCreatedAt ^^ [2024-01-15T10:00:00+02:00]'`. Both give `["Pancakes"]`. - **Remove ignored clauses.** Run `qk run configuration-remove-ignored --config remove-ignored --filter 'Rating > 1 && Nope == 1'`. Exit `0`. Both targets give all four recipes. The `expression` has no `True == True`. - **Small parse limits.** Run `qk run configuration-small-limits --config small-limits --filter '((((Title == "Pancakes"))))'`. Exit `2`. Both targets have `error.type` `QueryKit.Exceptions.QueryKitNestingDepthExceededException`. - **Quoted parentheses and the nesting depth.** Run `qk run configuration-nesting-depth --config depth-10 --filter 'Title == "))))))))))))))))))))" || ((((((((((((((((((((Title == "Pancakes"))))))))))))))))))))'`. Exit `2`. Both targets have `error.type` `QueryKit.Exceptions.QueryKitNestingDepthExceededException` with the message `The filter has a nesting depth of 11, which exceeds the maximum allowed depth of 10.` diff --git a/.agents/skills/verify-querykit/harness/Driver/Configs.cs b/.agents/skills/verify-querykit/harness/Driver/Configs.cs index e251a04..a1aab79 100644 --- a/.agents/skills/verify-querykit/harness/Driver/Configs.cs +++ b/.agents/skills/verify-querykit/harness/Driver/Configs.cs @@ -80,6 +80,16 @@ public static class Configs ["parameterized"] = ("ParameterizeFilterValues = true.", () => new QueryKitConfiguration(s => s.ParameterizeFilterValues = true)), + ["unspecified-datetime"] = ("DateTimeKindForValuesWithoutOffset = Unspecified.", + () => new QueryKitConfiguration(s => s.DateTimeKindForValuesWithoutOffset = DateTimeKind.Unspecified)), + + ["unspecified-datetime-parameterized"] = ("DateTimeKindForValuesWithoutOffset = Unspecified, ParameterizeFilterValues = true.", + () => new QueryKitConfiguration(s => + { + s.DateTimeKindForValuesWithoutOffset = DateTimeKind.Unspecified; + s.ParameterizeFilterValues = true; + })), + ["remove-ignored"] = ("IgnoredClauseBehavior = Remove, AllowUnknownProperties = true, Rating PreventFilter.", () => new QueryKitConfiguration(s => { diff --git a/.agents/skills/verify-querykit/harness/Driver/Model.cs b/.agents/skills/verify-querykit/harness/Driver/Model.cs index a4ff3ca..7791a41 100644 --- a/.agents/skills/verify-querykit/harness/Driver/Model.cs +++ b/.agents/skills/verify-querykit/harness/Driver/Model.cs @@ -32,6 +32,10 @@ public class Recipe public bool IsVegetarian { get; set; } public Visibility Visibility { get; set; } public DateTime CreatedAt { get; set; } + + // The same wall-clock time as CreatedAt, in a column without a time zone. + [System.ComponentModel.DataAnnotations.Schema.Column(TypeName = "timestamp without time zone")] + public DateTime LocalCreatedAt { get; set; } public DateOnly? DateOfOrigin { get; set; } public string Sku { get; set; } = ""; public string? Serving { get; set; } @@ -99,6 +103,7 @@ private static Recipe Recipe(int n, string title, Author author, int rating, dec IsVegetarian = vegetarian, Visibility = visibility, CreatedAt = created, + LocalCreatedAt = DateTime.SpecifyKind(created, DateTimeKind.Unspecified), DateOfOrigin = origin, Sku = sku, Serving = serving, diff --git a/QueryKit.IntegrationTests/Tests/FilterParsingRegressionTests.cs b/QueryKit.IntegrationTests/Tests/FilterParsingRegressionTests.cs index db9ef56..a2269c3 100644 --- a/QueryKit.IntegrationTests/Tests/FilterParsingRegressionTests.cs +++ b/QueryKit.IntegrationTests/Tests/FilterParsingRegressionTests.cs @@ -342,4 +342,102 @@ public void string_operator_with_null_value_throws_querykit_exception() // Assert act.Should().Throw(); } + + // Npgsql needs a Utc DateTime for a timestamptz column. A value without an offset is UTC by default. + [Theory] + [InlineData("SpecificDateTime == 2024-01-15T08:00:00", false)] + [InlineData("SpecificDateTime == 2024-01-15T08:00:00", true)] + [InlineData("SpecificDateTime == \"2024-01-15T08:00:00\"", false)] + [InlineData("SpecificDateTime == \"2024-01-15T08:00:00\"", true)] + [InlineData("SpecificDateTime == 2024-01-15T10:00:00+02:00", false)] + [InlineData("SpecificDateTime == 2024-01-15T10:00:00+02:00", true)] + [InlineData("SpecificDateTime ^^ [2024-01-15T08:00:00]", false)] + [InlineData("SpecificDateTime ^^ [2024-01-15T08:00:00]", true)] + [InlineData("SpecificDateTime ^^ [\"2024-01-15T08:00:00\"]", false)] + [InlineData("SpecificDateTime ^^ [\"2024-01-15T08:00:00\"]", true)] + [InlineData("SpecificDateTime ^^ [2024-01-15T10:00:00+02:00]", false)] + [InlineData("SpecificDateTime ^^ [2024-01-15T10:00:00+02:00]", true)] + [InlineData("SpecificDate == 2024-01-15T08:00:00", false)] + [InlineData("SpecificDate == 2024-01-15T08:00:00", true)] + [InlineData("SpecificDate == \"2024-01-15T08:00:00\"", false)] + [InlineData("SpecificDate == \"2024-01-15T08:00:00\"", true)] + [InlineData("SpecificDate == 2024-01-15T10:00:00+02:00", false)] + [InlineData("SpecificDate == 2024-01-15T10:00:00+02:00", true)] + [InlineData("SpecificDate ^^ [2024-01-15T08:00:00]", false)] + [InlineData("SpecificDate ^^ [2024-01-15T08:00:00]", true)] + [InlineData("SpecificDate ^^ [\"2024-01-15T08:00:00\"]", false)] + [InlineData("SpecificDate ^^ [\"2024-01-15T08:00:00\"]", true)] + [InlineData("SpecificDate ^^ [2024-01-15T10:00:00+02:00]", false)] + [InlineData("SpecificDate ^^ [2024-01-15T10:00:00+02:00]", true)] + public async Task timestamptz_value_without_offset_is_utc_by_default(string valueFilter, bool parameterize) + { + // Arrange + var testingServiceScope = new TestingServiceScope(); + var title = $"utc {Guid.NewGuid()}"; + var fakePersonOne = new FakeTestingPersonBuilder() + .WithTitle(title) + .WithSpecificDateTime(new DateTime(2024, 1, 15, 8, 0, 0, DateTimeKind.Utc)) + .WithSpecificDate(new DateTimeOffset(2024, 1, 15, 8, 0, 0, TimeSpan.Zero)) + .Build(); + var fakePersonTwo = new FakeTestingPersonBuilder() + .WithTitle(title) + .WithSpecificDateTime(new DateTime(2024, 1, 15, 9, 0, 0, DateTimeKind.Utc)) + .WithSpecificDate(new DateTimeOffset(2024, 1, 15, 9, 0, 0, TimeSpan.Zero)) + .Build(); + await testingServiceScope.InsertAsync(fakePersonOne, fakePersonTwo); + + var input = $"""{nameof(TestingPerson.Title)} == "{title}" && {valueFilter}"""; + var config = new QueryKitConfiguration(settings => settings.ParameterizeFilterValues = parameterize); + + // Act + var queryablePeople = testingServiceScope.DbContext().People; + var people = await queryablePeople.ApplyQueryKitFilter(input, config).ToListAsync(); + + // Assert + people.Select(x => x.Id).Should().Equal(fakePersonOne.Id); + } + + // Npgsql rejects a Utc DateTime for a timestamp without time zone column. Unspecified keeps the wall-clock time. + [Theory] + [InlineData("LocalDateTime == 2024-01-15T08:00:00", false)] + [InlineData("LocalDateTime == 2024-01-15T08:00:00", true)] + [InlineData("LocalDateTime == \"2024-01-15T08:00:00\"", false)] + [InlineData("LocalDateTime == \"2024-01-15T08:00:00\"", true)] + [InlineData("LocalDateTime == 2024-01-15T10:00:00+02:00", false)] + [InlineData("LocalDateTime == 2024-01-15T10:00:00+02:00", true)] + [InlineData("LocalDateTime ^^ [2024-01-15T08:00:00]", false)] + [InlineData("LocalDateTime ^^ [2024-01-15T08:00:00]", true)] + [InlineData("LocalDateTime ^^ [\"2024-01-15T08:00:00\"]", false)] + [InlineData("LocalDateTime ^^ [\"2024-01-15T08:00:00\"]", true)] + [InlineData("LocalDateTime ^^ [2024-01-15T10:00:00+02:00]", false)] + [InlineData("LocalDateTime ^^ [2024-01-15T10:00:00+02:00]", true)] + public async Task timestamp_without_time_zone_value_matches_with_unspecified_kind(string valueFilter, bool parameterize) + { + // Arrange + var testingServiceScope = new TestingServiceScope(); + var title = $"unspecified {Guid.NewGuid()}"; + var fakePersonOne = new FakeTestingPersonBuilder() + .WithTitle(title) + .WithLocalDateTime(new DateTime(2024, 1, 15, 8, 0, 0, DateTimeKind.Unspecified)) + .Build(); + var fakePersonTwo = new FakeTestingPersonBuilder() + .WithTitle(title) + .WithLocalDateTime(new DateTime(2024, 1, 15, 9, 0, 0, DateTimeKind.Unspecified)) + .Build(); + await testingServiceScope.InsertAsync(fakePersonOne, fakePersonTwo); + + var input = $"""{nameof(TestingPerson.Title)} == "{title}" && {valueFilter}"""; + var config = new QueryKitConfiguration(settings => + { + settings.ParameterizeFilterValues = parameterize; + settings.DateTimeKindForValuesWithoutOffset = DateTimeKind.Unspecified; + }); + + // Act + var queryablePeople = testingServiceScope.DbContext().People; + var people = await queryablePeople.ApplyQueryKitFilter(input, config).ToListAsync(); + + // Assert + people.Select(x => x.Id).Should().Equal(fakePersonOne.Id); + } } diff --git a/QueryKit.IntegrationTests/Tests/InListQueryCacheTests.cs b/QueryKit.IntegrationTests/Tests/InListQueryCacheTests.cs index a37f5f9..f03f414 100644 --- a/QueryKit.IntegrationTests/Tests/InListQueryCacheTests.cs +++ b/QueryKit.IntegrationTests/Tests/InListQueryCacheTests.cs @@ -29,40 +29,26 @@ public void equal_in_lists_compile_once(string input) cache.Compilations.Should().Be(1); } - // DateTime.Equals ignores Kind. If one cache entry served both requests, the first request would - // decide the result of the second. + // A list value without a zone is read as UTC, so it is the same value as a list value with the UTC zone. + // Both requests filter, and they share one compiled query. [Theory] - [InlineData("^^", true)] - [InlineData("^^*", true)] - [InlineData("!^^", true)] - [InlineData("!^^*", true)] - [InlineData("^^", false)] - [InlineData("^^*", false)] - [InlineData("!^^", false)] - [InlineData("!^^*", false)] - public async Task datetime_in_lists_that_differ_only_in_kind_do_not_share_a_query(string op, bool utcFirst) + [InlineData("^^")] + [InlineData("^^*")] + [InlineData("!^^")] + [InlineData("!^^*")] + public async Task datetime_in_list_without_a_zone_filters_like_the_utc_zone(string op) { var testingServiceScope = new TestingServiceScope(); var when = new DateTime(2001, 1, 1, 0, 0, 0, DateTimeKind.Utc).AddSeconds(Random.Shared.Next(1, 900_000_000)); var person = new FakeTestingPersonBuilder().WithSpecificDateTime(when).Build(); await testingServiceScope.InsertAsync(person); var iso = when.ToString("yyyy-MM-ddTHH:mm:ss"); - var utc = $"""SpecificDateTime {op} ["{iso}Z"]"""; - var noZone = $"""SpecificDateTime {op} ["{iso}"]"""; var cache = new QueryCache(testingServiceScope); - if (utcFirst) - { - await UtcRequestFilters(cache, utc, op, person.Id); - await NoZoneRequestFails(cache, noZone); - } - else - { - await NoZoneRequestFails(cache, noZone); - await UtcRequestFilters(cache, utc, op, person.Id); - } + await RequestFilters(cache, $"""SpecificDateTime {op} ["{iso}"]""", op, person.Id); + await RequestFilters(cache, $"""SpecificDateTime {op} ["{iso}Z"]""", op, person.Id); - cache.Compilations.Should().Be(2); + cache.Compilations.Should().Be(1); } // The hash of an array covers only its last 8 items. Lists that differ before the last 8 items @@ -84,20 +70,13 @@ public void long_in_lists_with_the_same_last_items_get_different_cache_keys(stri cache.KeyHashCodes.Distinct().Should().HaveCount(2); } - private static async Task UtcRequestFilters(QueryCache cache, string input, string op, Guid personId) + private static async Task RequestFilters(QueryCache cache, string input, string op, Guid personId) { var ids = await cache.Ids(input); ids.Contains(personId).Should().Be(!op.StartsWith('!')); } - private static async Task NoZoneRequestFails(QueryCache cache, string input) - { - var act = () => cache.Ids(input); - - await act.Should().ThrowAsync().WithMessage("*Unspecified DateTime*"); - } - private sealed class QueryCache { private readonly DbContextOptions _options; diff --git a/QueryKit.UnitTests/AliasCultureTests.cs b/QueryKit.UnitTests/AliasCultureTests.cs index ea5919e..b455c60 100644 --- a/QueryKit.UnitTests/AliasCultureTests.cs +++ b/QueryKit.UnitTests/AliasCultureTests.cs @@ -58,7 +58,7 @@ public void query_name_does_not_match_with_the_dotted_and_dotless_i_rules_of_tr_ { var config = new QueryKitConfiguration(settings => { - settings.Property(x => x.Rating).HasQueryName(queryName); + settings.Property(x => x.Rating!).HasQueryName(queryName); }); var turkish = () => WithCulture("tr-TR", () => FilterParser.ParseFilter(input, config)); diff --git a/QueryKit.UnitTests/DateTimeKindForValuesWithoutOffsetTests.cs b/QueryKit.UnitTests/DateTimeKindForValuesWithoutOffsetTests.cs new file mode 100644 index 0000000..98d920a --- /dev/null +++ b/QueryKit.UnitTests/DateTimeKindForValuesWithoutOffsetTests.cs @@ -0,0 +1,202 @@ +namespace QueryKit.UnitTests; + +using System.Linq.Expressions; +using Configuration; +using FluentAssertions; +using WebApiTestProject.Entities; + +// Npgsql needs a Utc DateTime for a timestamptz column and rejects a Utc DateTime for a timestamp without +// time zone column. DateTimeKindForValuesWithoutOffset sets the kind of a value that has no offset. +public class DateTimeKindForValuesWithoutOffsetTests +{ + private static readonly DateTime EightUtc = new(2024, 1, 15, 8, 0, 0, DateTimeKind.Utc); + + public static TheoryData DateTimeInputs => new() + { + "SpecificDateTime == 2024-01-15T08:00:00", + "SpecificDateTime == \"2024-01-15T08:00:00\"", + "SpecificDateTime ^^ [2024-01-15T08:00:00]", + "SpecificDateTime ^^ [\"2024-01-15T08:00:00\"]", + "stamp == 2024-01-15T08:00:00", + }; + + [Theory] + [MemberData(nameof(DateTimeInputs))] + public void value_without_offset_is_utc_by_default(string input) + { + foreach (var parameterize in new[] { false, true }) + { + var config = Config(parameterize, kind: null); + + DateTimesIn(input, config).Should().Equal((EightUtc.Ticks, DateTimeKind.Utc)); + } + } + + [Theory] + [MemberData(nameof(DateTimeInputs))] + public void value_without_offset_is_unspecified_when_configured(string input) + { + foreach (var parameterize in new[] { false, true }) + { + var config = Config(parameterize, DateTimeKind.Unspecified); + + DateTimesIn(input, config).Should().Equal((EightUtc.Ticks, DateTimeKind.Unspecified)); + } + } + + [Theory] + [MemberData(nameof(DateTimeInputs))] + public void value_without_offset_is_local_when_configured(string input) + { + foreach (var parameterize in new[] { false, true }) + { + var config = Config(parameterize, DateTimeKind.Local); + + DateTimesIn(input, config).Should().Equal((EightUtc.Ticks, DateTimeKind.Local)); + } + } + + [Theory] + [InlineData("SpecificDateTime == 2024-01-15T10:00:00+02:00")] + [InlineData("SpecificDateTime == 2024-01-15T08:00:00Z")] + [InlineData("SpecificDateTime ^^ [2024-01-15T10:00:00+02:00]")] + [InlineData("stamp == 2024-01-15T10:00:00+02:00")] + public void value_with_offset_is_the_utc_time_when_unspecified_is_configured(string input) + { + var config = Config(parameterize: false, DateTimeKind.Unspecified); + + DateTimesIn(input, config).Should().Equal((EightUtc.Ticks, DateTimeKind.Unspecified)); + } + + [Fact] + public void value_with_offset_is_the_local_time_when_local_is_configured() + { + var config = Config(parameterize: false, DateTimeKind.Local); + var expected = EightUtc.ToLocalTime(); + + DateTimesIn("SpecificDateTime == 2024-01-15T08:00:00Z", config).Should().Equal((expected.Ticks, DateTimeKind.Local)); + } + + [Theory] + [InlineData(null, "2024-01-15T08:00:00+00:00")] + [InlineData(DateTimeKind.Utc, "2024-01-15T08:00:00+00:00")] + [InlineData(DateTimeKind.Unspecified, "2024-01-15T08:00:00+00:00")] + public void date_time_offset_without_offset_is_utc_unless_local_is_configured(DateTimeKind? kind, string expected) + { + foreach (var input in new[] { "SpecificDate == 2024-01-15T08:00:00", "SpecificDate ^^ [2024-01-15T08:00:00]" }) + { + var values = ValuesIn(FilterParser.ParseFilter(input, Config(parameterize: false, kind))); + + values.Should().Equal(DateTimeOffset.Parse(expected)); + values.Cast().Should().OnlyContain(x => x.Offset == TimeSpan.Zero); + } + } + + [Fact] + public void date_time_offset_without_offset_is_local_when_local_is_configured() + { + var expected = new DateTimeOffset(DateTime.SpecifyKind(EightUtc, DateTimeKind.Local)).ToUniversalTime(); + + foreach (var input in new[] { "SpecificDate == 2024-01-15T08:00:00", "SpecificDate ^^ [2024-01-15T08:00:00]" }) + { + var values = ValuesIn(FilterParser.ParseFilter(input, Config(parameterize: false, DateTimeKind.Local))); + + values.Should().Equal(expected); + values.Cast().Should().OnlyContain(x => x.Offset == TimeSpan.Zero); + } + } + + [Fact] + public void setting_applies_only_to_the_filter_that_it_configures() + { + DateTimesIn("SpecificDateTime == 2024-01-15T08:00:00", Config(parameterize: false, DateTimeKind.Unspecified)); + + DateTimesIn("SpecificDateTime == 2024-01-15T08:00:00", config: null).Should().Equal((EightUtc.Ticks, DateTimeKind.Utc)); + } + + [Fact] + public void configuration_without_the_setting_uses_utc() + { + IQueryKitFilterBehavior behavior = new FilterBehaviorWithoutTheSetting(); + + behavior.DateTimeKindForValuesWithoutOffset.Should().Be(DateTimeKind.Utc); + } + + private static QueryKitConfiguration Config(bool parameterize, DateTimeKind? kind) + => new(settings => + { + settings.ParameterizeFilterValues = parameterize; + if (kind is not null) + { + settings.DateTimeKindForValuesWithoutOffset = kind.Value; + } + settings.CustomOperation((x, op, value) => x.SpecificDateTime == (DateTime)value) + .HasQueryName("stamp"); + }); + + private static List<(long Ticks, DateTimeKind Kind)> DateTimesIn(string input, QueryKitConfiguration? config) + => ValuesIn(FilterParser.ParseFilter(input, config)) + .Cast() + .Select(x => (x.Ticks, x.Kind)) + .ToList(); + + private static List ValuesIn(Expression expression) + { + var collector = new DateValueCollector(); + collector.Visit(expression); + return collector.Values; + } + + // Collects the date values of a filter, both from a literal and from a parameter. + private sealed class DateValueCollector : ExpressionVisitor + { + public List Values { get; } = []; + + protected override Expression VisitNew(NewExpression node) + { + if (node.Type != typeof(DateTime) && node.Type != typeof(DateTimeOffset)) + { + return base.VisitNew(node); + } + + Add(Expression.Lambda(node).Compile().DynamicInvoke()); + return node; + } + + protected override Expression VisitConstant(ConstantExpression node) + { + Add(node.Value); + return node; + } + + protected override Expression VisitMember(MemberExpression node) + { + if (node.Expression is not ConstantExpression) + { + return base.VisitMember(node); + } + + Add(Expression.Lambda(node).Compile().DynamicInvoke()); + return node; + } + + private void Add(object? value) + { + switch (value) + { + case DateTime or DateTimeOffset: + Values.Add(value); + break; + case System.Collections.IEnumerable list and not string: + Values.AddRange(list.Cast().Where(x => x is DateTime or DateTimeOffset)); + break; + } + } + } + + private sealed class FilterBehaviorWithoutTheSetting : IQueryKitFilterBehavior + { + public bool ParameterizeFilterValues => false; + public IgnoredClauseBehavior IgnoredClauseBehavior => IgnoredClauseBehavior.Remove; + } +} diff --git a/QueryKit.UnitTests/FilterParserTests.cs b/QueryKit.UnitTests/FilterParserTests.cs index 2681f88..0fbf54b 100644 --- a/QueryKit.UnitTests/FilterParserTests.cs +++ b/QueryKit.UnitTests/FilterParserTests.cs @@ -293,7 +293,7 @@ public void can_handle_datetime() { var input = """SpecificDateTime == 2022-07-01T00:00:03"""; var filterExpression = FilterParser.ParseFilter(input); - filterExpression.ToDisplayString().Should().Be(""""x => (x.SpecificDateTime == new DateTime(637922304030000000, Local))""""); + filterExpression.ToDisplayString().Should().Be(""""x => (x.SpecificDateTime == new DateTime(637922304030000000, Utc))""""); } [Fact] @@ -320,7 +320,7 @@ public void can_handle_datetime_another() { var input = """SpecificDateTime == "2022-07-01T00:00:03" """; var filterExpression = FilterParser.ParseFilter(input); - filterExpression.ToDisplayString().Should().Be(""""x => (x.SpecificDateTime == new DateTime(637922304030000000, Local))""""); + filterExpression.ToDisplayString().Should().Be(""""x => (x.SpecificDateTime == new DateTime(637922304030000000, Utc))""""); } [Fact] @@ -368,7 +368,7 @@ public void can_handle_datetime_with_milliseconds() { var input = """SpecificDateTime == 2022-07-01T00:00:03.123"""; var filterExpression = FilterParser.ParseFilter(input); - filterExpression.ToDisplayString().Should().Be("x => (x.SpecificDateTime == new DateTime(637922304031230000, Local))"); + filterExpression.ToDisplayString().Should().Be("x => (x.SpecificDateTime == new DateTime(637922304031230000, Utc))"); } [Fact] diff --git a/QueryKit.UnitTests/FilterParsingRegressionTests.cs b/QueryKit.UnitTests/FilterParsingRegressionTests.cs index f70e44d..707da62 100644 --- a/QueryKit.UnitTests/FilterParsingRegressionTests.cs +++ b/QueryKit.UnitTests/FilterParsingRegressionTests.cs @@ -362,6 +362,49 @@ public void string_operator_with_null_value_throws_querykit_exception(string inp act.Should().Throw(); } + [Theory] + [InlineData("SpecificDateTime == 2024-01-15T08:00:00", "x => (x.SpecificDateTime == new DateTime(638409024000000000, Utc))")] + [InlineData("SpecificDateTime == \"2024-01-15T08:00:00\"", "x => (x.SpecificDateTime == new DateTime(638409024000000000, Utc))")] + [InlineData("SpecificDateTime == 2024-01-15T10:00:00+02:00", "x => (x.SpecificDateTime == new DateTime(638409024000000000, Utc))")] + [InlineData("SpecificDateTime == 2024-01-15T08:00:00Z", "x => (x.SpecificDateTime == new DateTime(638409024000000000, Utc))")] + [InlineData("SpecificDate == 2024-01-15T08:00:00", "x => (x.SpecificDate == new Nullable`1(new DateTimeOffset(638409024000000000, 00:00:00)))")] + public void date_time_without_offset_is_utc(string input, string expected) + { + var filterExpression = FilterParser.ParseFilter(input); + + filterExpression.ToDisplayString().Should().Be(expected); + } + + [Theory] + [InlineData("SpecificDateTime ^^ [2024-01-15T08:00:00]")] + [InlineData("SpecificDateTime ^^ [2024-01-15T10:00:00+02:00]")] + [InlineData("SpecificDate ^^ [2024-01-15T08:00:00]")] + [InlineData("SpecificDate ^^ [2024-01-15T10:00:00+02:00]")] + public void date_time_list_value_matches_scalar_value(string input) + { + var people = new[] + { + new TestingPerson + { + Title = "match", + SpecificDateTime = new DateTime(2024, 1, 15, 8, 0, 0, DateTimeKind.Utc), + SpecificDate = new DateTimeOffset(2024, 1, 15, 8, 0, 0, TimeSpan.Zero), + }, + new TestingPerson + { + Title = "other", + SpecificDateTime = new DateTime(2024, 1, 15, 9, 0, 0, DateTimeKind.Utc), + SpecificDate = new DateTimeOffset(2024, 1, 15, 9, 0, 0, TimeSpan.Zero), + }, + }; + + var result = people.AsQueryable().ApplyQueryKitFilter(input).ToList(); + var scalarResult = people.AsQueryable().ApplyQueryKitFilter(input.Replace("^^ [", "== ").TrimEnd(']')).ToList(); + + result.Select(x => x.Title).Should().Equal("match"); + scalarResult.Select(x => x.Title).Should().Equal("match"); + } + private static TResult WithCulture(string cultureName, Func action) { var originalCulture = CultureInfo.CurrentCulture; diff --git a/QueryKit.UnitTests/ListDateCultureTests.cs b/QueryKit.UnitTests/ListDateCultureTests.cs index e9e072f..4550033 100644 --- a/QueryKit.UnitTests/ListDateCultureTests.cs +++ b/QueryKit.UnitTests/ListDateCultureTests.cs @@ -36,7 +36,7 @@ public void date_time_offset_in_a_list_matches_in_every_culture(string cultureNa var matches = WithCulture(cultureName, () => Matches(person, "SpecificDate ^^ [2024-01-15T10:00:00+02:00, 2024-01-16]")); var dateOnlyMatches = WithCulture(cultureName, () => Matches( - new TestingPerson { SpecificDate = DateTimeOffset.Parse("2024-01-16", CultureInfo.InvariantCulture) }, + new TestingPerson { SpecificDate = new DateTimeOffset(2024, 1, 16, 0, 0, 0, TimeSpan.Zero) }, "SpecificDate ^^ [2024-01-16]")); matches.Should().BeTrue(); diff --git a/QueryKit.WebApiTestProject/Database/PersonConfiguration.cs b/QueryKit.WebApiTestProject/Database/PersonConfiguration.cs index 9ee9289..56e8665 100644 --- a/QueryKit.WebApiTestProject/Database/PersonConfiguration.cs +++ b/QueryKit.WebApiTestProject/Database/PersonConfiguration.cs @@ -16,6 +16,9 @@ public void Configure(EntityTypeBuilder builder) .IsRequired(false); builder.Property(x => x.Nickname); + + builder.Property(x => x.LocalDateTime) + .HasColumnType("timestamp without time zone"); builder.OwnsOne(x => x.PhysicalAddress, opts => { diff --git a/QueryKit.WebApiTestProject/Entities/TestingPerson.cs b/QueryKit.WebApiTestProject/Entities/TestingPerson.cs index 50176eb..3e9fbdb 100644 --- a/QueryKit.WebApiTestProject/Entities/TestingPerson.cs +++ b/QueryKit.WebApiTestProject/Entities/TestingPerson.cs @@ -12,6 +12,7 @@ public class TestingPerson public bool? Favorite { get; set; } public DateTimeOffset? SpecificDate { get; set; } public DateTime SpecificDateTime { get; set; } + public DateTime? LocalDateTime { get; set; } public TimeOnly? Time { get; set; } public Guid Id { get; set; } = Guid.NewGuid(); public EmailAddress Email { get; set; } = null!; diff --git a/QueryKit.WebApiTestProject/Migrations/20261003115007_AddPersonLocalDateTime.Designer.cs b/QueryKit.WebApiTestProject/Migrations/20261003115007_AddPersonLocalDateTime.Designer.cs new file mode 100644 index 0000000..4fb5987 --- /dev/null +++ b/QueryKit.WebApiTestProject/Migrations/20261003115007_AddPersonLocalDateTime.Designer.cs @@ -0,0 +1,363 @@ +// +using System; +using System.Collections.Generic; +using Microsoft.EntityFrameworkCore; +using Microsoft.EntityFrameworkCore.Infrastructure; +using Microsoft.EntityFrameworkCore.Migrations; +using Microsoft.EntityFrameworkCore.Storage.ValueConversion; +using Npgsql.EntityFrameworkCore.PostgreSQL.Metadata; +using QueryKit.WebApiTestProject.Database; + +#nullable disable + +namespace QueryKit.WebApiTestProject.Migrations +{ + [DbContext(typeof(TestingDbContext))] + [Migration("20261003115007_AddPersonLocalDateTime")] + partial class AddPersonLocalDateTime + { + /// + protected override void BuildTargetModel(ModelBuilder modelBuilder) + { +#pragma warning disable 612, 618 + modelBuilder + .HasAnnotation("ProductVersion", "10.0.1") + .HasAnnotation("Relational:MaxIdentifierLength", 63); + + NpgsqlModelBuilderExtensions.HasPostgresExtension(modelBuilder, "fuzzystrmatch"); + NpgsqlModelBuilderExtensions.UseIdentityByDefaultColumns(modelBuilder); + + modelBuilder.HasSequence("AUT") + .StartsAt(100045702L); + + modelBuilder.Entity("QueryKit.WebApiTestProject.Entities.Authors.Author", b => + { + b.Property("Id") + .ValueGeneratedOnAdd() + .HasColumnType("uuid") + .HasColumnName("id"); + + b.Property("InternalIdentifier") + .IsRequired() + .ValueGeneratedOnAdd() + .HasColumnType("text") + .HasColumnName("internal_identifier") + .HasDefaultValueSql("concat('AUT', nextval('\"AUT\"'))"); + + b.Property("Name") + .IsRequired() + .HasColumnType("text") + .HasColumnName("name"); + + b.Property("RecipeId") + .HasColumnType("uuid") + .HasColumnName("recipe_id"); + + b.HasKey("Id") + .HasName("pk_authors"); + + b.HasIndex("RecipeId") + .IsUnique() + .HasDatabaseName("ix_authors_recipe_id"); + + b.ToTable("authors", (string)null); + }); + + modelBuilder.Entity("QueryKit.WebApiTestProject.Entities.Ingredients.Ingredient", b => + { + b.Property("Id") + .ValueGeneratedOnAdd() + .HasColumnType("uuid") + .HasColumnName("id"); + + b.Property("ExpiresOn") + .HasColumnType("timestamp with time zone") + .HasColumnName("expires_on"); + + b.Property("Measure") + .IsRequired() + .HasColumnType("text") + .HasColumnName("measure"); + + b.Property("MinimumQuality") + .HasColumnType("integer") + .HasColumnName("minimum_quality"); + + b.Property("Name") + .IsRequired() + .HasColumnType("text") + .HasColumnName("name"); + + b.Property("QualityLevel") + .HasColumnType("bigint") + .HasColumnName("quality_level"); + + b.Property("Quantity") + .IsRequired() + .HasColumnType("text") + .HasColumnName("quantity"); + + b.Property("RecipeId") + .HasColumnType("uuid") + .HasColumnName("recipe_id"); + + b.HasKey("Id") + .HasName("pk_ingredients"); + + b.HasIndex("RecipeId") + .HasDatabaseName("ix_ingredients_recipe_id"); + + b.ToTable("ingredients", (string)null); + }); + + modelBuilder.Entity("QueryKit.WebApiTestProject.Entities.Ingredients.IngredientPreparation", b => + { + b.Property("Id") + .ValueGeneratedOnAdd() + .HasColumnType("uuid") + .HasColumnName("id"); + + b.Property("IngredientId") + .HasColumnType("uuid") + .HasColumnName("ingredient_id"); + + b.Property("Text") + .IsRequired() + .HasColumnType("text") + .HasColumnName("text"); + + b.HasKey("Id") + .HasName("pk_ingredient_preparations"); + + b.HasIndex("IngredientId") + .HasDatabaseName("ix_ingredient_preparations_ingredient_id"); + + b.ToTable("ingredient_preparations", (string)null); + }); + + modelBuilder.Entity("QueryKit.WebApiTestProject.Entities.Recipes.Recipe", b => + { + b.Property("Id") + .ValueGeneratedOnAdd() + .HasColumnType("uuid") + .HasColumnName("id"); + + b.Property("DateOfOrigin") + .HasColumnType("date") + .HasColumnName("date_of_origin"); + + b.Property("Directions") + .IsRequired() + .HasColumnType("text") + .HasColumnName("directions"); + + b.Property("HaveMadeItMyself") + .HasColumnType("boolean") + .HasColumnName("have_made_it_myself"); + + b.Property("Rating") + .HasColumnType("integer") + .HasColumnName("rating"); + + b.Property("SecondaryId") + .HasColumnType("uuid") + .HasColumnName("secondary_id"); + + b.PrimitiveCollection>("Tags") + .IsRequired() + .HasColumnType("text[]") + .HasColumnName("tags"); + + b.Property("Title") + .IsRequired() + .HasColumnType("text") + .HasColumnName("title"); + + b.Property("Visibility") + .IsRequired() + .HasColumnType("text") + .HasColumnName("visibility"); + + b.ComplexProperty(typeof(Dictionary), "CollectionEmail", "QueryKit.WebApiTestProject.Entities.Recipes.Recipe.CollectionEmail#EmailAddress", b1 => + { + b1.IsRequired(); + + b1.Property("Value") + .HasColumnType("text") + .HasColumnName("collection_email"); + }); + + b.HasKey("Id") + .HasName("pk_recipes"); + + b.ToTable("recipes", (string)null); + }); + + modelBuilder.Entity("QueryKit.WebApiTestProject.Entities.TestingPerson", b => + { + b.Property("Id") + .ValueGeneratedOnAdd() + .HasColumnType("uuid") + .HasColumnName("id"); + + b.Property("Age") + .HasColumnType("integer") + .HasColumnName("age"); + + b.Property("BirthMonth") + .HasColumnType("integer") + .HasColumnName("birth_month"); + + b.Property("Date") + .HasColumnType("date") + .HasColumnName("date"); + + b.Property("Email") + .HasColumnType("text") + .HasColumnName("email"); + + b.Property("Favorite") + .HasColumnType("boolean") + .HasColumnName("favorite"); + + b.Property("FirstName") + .HasColumnType("text") + .HasColumnName("first_name"); + + b.Property("LastName") + .HasColumnType("text") + .HasColumnName("last_name"); + + b.Property("LocalDateTime") + .HasColumnType("timestamp without time zone") + .HasColumnName("local_date_time"); + + b.Property("Nickname") + .HasColumnType("text") + .HasColumnName("nickname"); + + b.Property("Rating") + .HasColumnType("numeric") + .HasColumnName("rating"); + + b.Property("SpecificDate") + .HasColumnType("timestamp with time zone") + .HasColumnName("specific_date"); + + b.Property("SpecificDateTime") + .HasColumnType("timestamp with time zone") + .HasColumnName("specific_date_time"); + + b.Property("Time") + .HasColumnType("time without time zone") + .HasColumnName("time"); + + b.Property("Title") + .HasColumnType("text") + .HasColumnName("title"); + + b.HasKey("Id") + .HasName("pk_people"); + + b.ToTable("people", (string)null); + }); + + modelBuilder.Entity("QueryKit.WebApiTestProject.Entities.Authors.Author", b => + { + b.HasOne("QueryKit.WebApiTestProject.Entities.Recipes.Recipe", "Recipe") + .WithOne("Author") + .HasForeignKey("QueryKit.WebApiTestProject.Entities.Authors.Author", "RecipeId") + .OnDelete(DeleteBehavior.Cascade) + .IsRequired() + .HasConstraintName("fk_authors_recipes_recipe_id"); + + b.Navigation("Recipe"); + }); + + modelBuilder.Entity("QueryKit.WebApiTestProject.Entities.Ingredients.Ingredient", b => + { + b.HasOne("QueryKit.WebApiTestProject.Entities.Recipes.Recipe", "Recipe") + .WithMany("Ingredients") + .HasForeignKey("RecipeId") + .OnDelete(DeleteBehavior.Cascade) + .IsRequired() + .HasConstraintName("fk_ingredients_recipes_recipe_id"); + + b.Navigation("Recipe"); + }); + + modelBuilder.Entity("QueryKit.WebApiTestProject.Entities.Ingredients.IngredientPreparation", b => + { + b.HasOne("QueryKit.WebApiTestProject.Entities.Ingredients.Ingredient", null) + .WithMany("Preparations") + .HasForeignKey("IngredientId") + .HasConstraintName("fk_ingredient_preparations_ingredients_ingredient_id"); + }); + + modelBuilder.Entity("QueryKit.WebApiTestProject.Entities.TestingPerson", b => + { + b.OwnsOne("QueryKit.WebApiTestProject.Entities.Address", "PhysicalAddress", b1 => + { + b1.Property("TestingPersonId") + .HasColumnType("uuid") + .HasColumnName("id"); + + b1.Property("City") + .IsRequired() + .HasColumnType("text") + .HasColumnName("physical_address_city"); + + b1.Property("Country") + .IsRequired() + .HasColumnType("text") + .HasColumnName("physical_address_country"); + + b1.Property("Line1") + .IsRequired() + .HasColumnType("text") + .HasColumnName("physical_address_line1"); + + b1.Property("Line2") + .IsRequired() + .HasColumnType("text") + .HasColumnName("physical_address_line2"); + + b1.Property("PostalCode") + .IsRequired() + .HasColumnType("text") + .HasColumnName("physical_address_postal_code"); + + b1.Property("State") + .IsRequired() + .HasColumnType("text") + .HasColumnName("physical_address_state"); + + b1.HasKey("TestingPersonId"); + + b1.ToTable("people"); + + b1.WithOwner() + .HasForeignKey("TestingPersonId") + .HasConstraintName("fk_people_people_id"); + }); + + b.Navigation("PhysicalAddress") + .IsRequired(); + }); + + modelBuilder.Entity("QueryKit.WebApiTestProject.Entities.Ingredients.Ingredient", b => + { + b.Navigation("Preparations"); + }); + + modelBuilder.Entity("QueryKit.WebApiTestProject.Entities.Recipes.Recipe", b => + { + b.Navigation("Author") + .IsRequired(); + + b.Navigation("Ingredients"); + }); +#pragma warning restore 612, 618 + } + } +} diff --git a/QueryKit.WebApiTestProject/Migrations/20261003115007_AddPersonLocalDateTime.cs b/QueryKit.WebApiTestProject/Migrations/20261003115007_AddPersonLocalDateTime.cs new file mode 100644 index 0000000..31b86d3 --- /dev/null +++ b/QueryKit.WebApiTestProject/Migrations/20261003115007_AddPersonLocalDateTime.cs @@ -0,0 +1,29 @@ +using System; +using Microsoft.EntityFrameworkCore.Migrations; + +#nullable disable + +namespace QueryKit.WebApiTestProject.Migrations +{ + /// + public partial class AddPersonLocalDateTime : Migration + { + /// + protected override void Up(MigrationBuilder migrationBuilder) + { + migrationBuilder.AddColumn( + name: "local_date_time", + table: "people", + type: "timestamp without time zone", + nullable: true); + } + + /// + protected override void Down(MigrationBuilder migrationBuilder) + { + migrationBuilder.DropColumn( + name: "local_date_time", + table: "people"); + } + } +} diff --git a/QueryKit.WebApiTestProject/Migrations/TestingDbContextModelSnapshot.cs b/QueryKit.WebApiTestProject/Migrations/TestingDbContextModelSnapshot.cs index 39faa82..25473af 100644 --- a/QueryKit.WebApiTestProject/Migrations/TestingDbContextModelSnapshot.cs +++ b/QueryKit.WebApiTestProject/Migrations/TestingDbContextModelSnapshot.cs @@ -225,6 +225,10 @@ protected override void BuildModel(ModelBuilder modelBuilder) .HasColumnType("text") .HasColumnName("last_name"); + b.Property("LocalDateTime") + .HasColumnType("timestamp without time zone") + .HasColumnName("local_date_time"); + b.Property("Nickname") .HasColumnType("text") .HasColumnName("nickname"); diff --git a/QueryKit/Configuration/IQueryKitFilterBehavior.cs b/QueryKit/Configuration/IQueryKitFilterBehavior.cs index 9011181..17dc370 100644 --- a/QueryKit/Configuration/IQueryKitFilterBehavior.cs +++ b/QueryKit/Configuration/IQueryKitFilterBehavior.cs @@ -2,11 +2,18 @@ namespace QueryKit.Configuration; /// /// The settings that control how the filter parser treats constant values and ignored clauses. A configuration -/// that does not implement this interface uses the defaults: values sent as constants, and -/// for an ignored clause. +/// that does not implement this interface uses the defaults: values sent as constants, +/// for an ignored clause, and UTC for a date and time +/// value without an offset. /// public interface IQueryKitFilterBehavior { bool ParameterizeFilterValues { get; } IgnoredClauseBehavior IgnoredClauseBehavior { get; } + + /// + /// The kind of a filter value that has no offset. See + /// . + /// + DateTimeKind DateTimeKindForValuesWithoutOffset => DateTimeKind.Utc; } diff --git a/QueryKit/Configuration/QueryKitConfiguration.cs b/QueryKit/Configuration/QueryKitConfiguration.cs index 55cec7e..33e33ec 100644 --- a/QueryKit/Configuration/QueryKitConfiguration.cs +++ b/QueryKit/Configuration/QueryKitConfiguration.cs @@ -73,6 +73,7 @@ public class QueryKitConfiguration : IQueryKitConfiguration, IQueryKitParseLimit public int MaxInputLength { get; set; } public CaseInsensitiveMode CaseInsensitiveComparison { get; set; } public bool ParameterizeFilterValues { get; set; } + public DateTimeKind DateTimeKindForValuesWithoutOffset { get; set; } public IgnoredClauseBehavior IgnoredClauseBehavior { get; set; } public QueryKitConfiguration(Action configureSettings) @@ -116,6 +117,7 @@ public QueryKitConfiguration(Action configureSettings) MaxInputLength = settings.MaxInputLength; CaseInsensitiveComparison = settings.CaseInsensitiveComparison; ParameterizeFilterValues = settings.ParameterizeFilterValues; + DateTimeKindForValuesWithoutOffset = settings.DateTimeKindForValuesWithoutOffset; IgnoredClauseBehavior = settings.IgnoredClauseBehavior; } } \ No newline at end of file diff --git a/QueryKit/Configuration/QueryKitSettings.cs b/QueryKit/Configuration/QueryKitSettings.cs index 0d4fe3f..fc0407a 100644 --- a/QueryKit/Configuration/QueryKitSettings.cs +++ b/QueryKit/Configuration/QueryKitSettings.cs @@ -56,6 +56,14 @@ public class QueryKitSettings /// public bool ParameterizeFilterValues { get; set; } + /// + /// The kind of a filter value that has no offset, for example 2024-01-15T08:00:00. + /// The default is , which Npgsql needs for a timestamp with time zone column. + /// Use for a timestamp without time zone column. Then a value with + /// an offset becomes its UTC time. reads the value in the time zone of the server. + /// + public DateTimeKind DateTimeKindForValuesWithoutOffset { get; set; } = DateTimeKind.Utc; + /// /// What the filter parser does with a clause on a prevented or unknown property. The default is /// . Use diff --git a/QueryKit/FilterParser.cs b/QueryKit/FilterParser.cs index db27e21..569800a 100644 --- a/QueryKit/FilterParser.cs +++ b/QueryKit/FilterParser.cs @@ -33,10 +33,12 @@ public static Expression> ParseFilter(string input, IQueryKitCo var nestingDepthBefore = _nestingDepth; var queryNameOverUnknownBefore = _queryNameOverUnknown; var queryNameFallbackOffBefore = _queryNameFallbackOff; + var dateTimeKindBefore = _dateTimeKindForValuesWithoutOffset; _maxNestingDepth = (config as IQueryKitParseLimits)?.MaxNestingDepth ?? QueryKitSettings.DefaultMaxNestingDepth; _nestingDepth = 0; _queryNameOverUnknown = false; _queryNameFallbackOff = false; + _dateTimeKindForValuesWithoutOffset = (config as IQueryKitFilterBehavior)?.DateTimeKindForValuesWithoutOffset; try { try @@ -75,6 +77,7 @@ public static Expression> ParseFilter(string input, IQueryKitCo _nestingDepth = nestingDepthBefore; _queryNameOverUnknown = queryNameOverUnknownBefore; _queryNameFallbackOff = queryNameFallbackOffBefore; + _dateTimeKindForValuesWithoutOffset = dateTimeKindBefore; } return Expression.Lambda>(expr, parameter); @@ -111,6 +114,9 @@ private static void EnsureWithinInputLength(string input, IQueryKitConfiguration [ThreadStatic] private static bool _queryNameOverUnknown; [ThreadStatic] private static bool _queryNameFallbackOff; + // The kind of a DateTime value without an offset in the parse on this thread. Null means the default, UTC. + [ThreadStatic] private static DateTimeKind? _dateTimeKindForValuesWithoutOffset; + // Parses '(' inner ')' and counts the group against MaxNestingDepth. The grammar does the count, // so a '(' or ')' inside a quoted value cannot change it. The parser recurses once for each group, // so the limit also limits the depth of the call stack. @@ -458,6 +464,37 @@ from trailingSpaces in Parse.WhiteSpace.Many() select LogicalOperator.GetByOperatorString(match.Operator))); } + private static DateTimeKind DateTimeKindForValuesWithoutOffset => _dateTimeKindForValuesWithoutOffset ?? DateTimeKind.Utc; + + // A date and time value without an offset gets the kind from DateTimeKindForValuesWithoutOffset. Utc, the default, + // does not depend on the server time zone. Unspecified keeps the wall-clock time for a timestamp without time zone + // column, and a value with an offset becomes its UTC time. Local reads the value in the server time zone. + private static DateTime ParseDateTime(string value) + => ToKindForValuesWithoutOffset(DateTime.Parse(value, CultureInfo.InvariantCulture, DateTimeStylesForValuesWithoutOffset())); + + private static bool TryParseDateTime(string value, out DateTime result) + { + var parsed = DateTime.TryParse(value, CultureInfo.InvariantCulture, DateTimeStylesForValuesWithoutOffset(), out result); + result = ToKindForValuesWithoutOffset(result); + return parsed; + } + + private static DateTimeStyles DateTimeStylesForValuesWithoutOffset() => DateTimeKindForValuesWithoutOffset switch + { + DateTimeKind.Local => DateTimeStyles.AssumeLocal, + DateTimeKind.Unspecified => DateTimeStyles.AdjustToUniversal, + _ => DateTimeStyles.AssumeUniversal | DateTimeStyles.AdjustToUniversal + }; + + private static DateTime ToKindForValuesWithoutOffset(DateTime value) + => DateTimeKindForValuesWithoutOffset == DateTimeKind.Unspecified ? DateTime.SpecifyKind(value, DateTimeKind.Unspecified) : value; + + // Npgsql only accepts a DateTimeOffset parameter with offset 0. The UTC value is the same instant. + private static DateTimeOffset ParseDateTimeOffset(string value) + => DateTimeOffset.Parse(value, CultureInfo.InvariantCulture, + DateTimeKindForValuesWithoutOffset == DateTimeKind.Local ? DateTimeStyles.AssumeLocal : DateTimeStyles.AssumeUniversal) + .ToUniversalTime(); + private static readonly Dictionary> TypeConversionFunctions = new() { { typeof(string), value => value }, @@ -471,8 +508,8 @@ from trailingSpaces in Parse.WhiteSpace.Many() { typeof(long), value => long.Parse(value, CultureInfo.InvariantCulture) }, { typeof(short), value => short.Parse(value, CultureInfo.InvariantCulture) }, { typeof(byte), value => byte.Parse(value, CultureInfo.InvariantCulture) }, - { typeof(DateTime), value => DateTime.Parse(value, CultureInfo.InvariantCulture, DateTimeStyles.AdjustToUniversal) }, - { typeof(DateTimeOffset), value => DateTimeOffset.Parse(value, CultureInfo.InvariantCulture).ToUniversalTime() }, + { typeof(DateTime), value => ParseDateTime(value) }, + { typeof(DateTimeOffset), value => ParseDateTimeOffset(value) }, { typeof(DateOnly), value => DateOnly.Parse(value, CultureInfo.InvariantCulture) }, { typeof(TimeOnly), value => TimeOnly.Parse(value, CultureInfo.InvariantCulture) }, { typeof(TimeSpan), value => TimeSpan.Parse(value) }, @@ -662,22 +699,16 @@ private static Expression CreateRightExprFromType(Type leftExprType, string righ if (targetType == typeof(DateTime)) { - var dtStyle = right.EndsWith("Z") ? DateTimeStyles.AdjustToUniversal : DateTimeStyles.AssumeLocal; - var dt = DateTime.Parse(right, CultureInfo.InvariantCulture, dtStyle); - if (right.EndsWith("Z")) - { - dt = DateTime.SpecifyKind(dt, DateTimeKind.Utc); - } + var dt = ParseDateTime(right); return FilterValue.Create(dt, rawType); } if (targetType == typeof(DateTimeOffset)) { - var dtStyle = right.EndsWith("Z") ? DateTimeStyles.AdjustToUniversal : DateTimeStyles.AssumeLocal; - var dto = DateTimeOffset.Parse(right, CultureInfo.InvariantCulture, dtStyle); - // Npgsql only accepts a DateTimeOffset parameter with offset 0. The UTC value is the same instant. - return FilterValue.Create(dto.ToUniversalTime(), rawType); + var dto = ParseDateTimeOffset(right); + + return FilterValue.Create(dto, rawType); } if (targetType == typeof(DateOnly)) @@ -1723,7 +1754,7 @@ private static Expression CreateCustomOperationExpression(ParameterExpression return doubleValue; // Try DateTime - if (DateTime.TryParse(value, CultureInfo.InvariantCulture, DateTimeStyles.AdjustToUniversal, out var dateTimeValue)) + if (TryParseDateTime(value, out var dateTimeValue)) return dateTimeValue; // Try Guid diff --git a/README.md b/README.md index 99e8ea8..60f7b05 100644 --- a/README.md +++ b/README.md @@ -255,6 +255,7 @@ Property list grouping is particularly useful for: * `DateTime`: `var filterInput = """Birthday == "2022-07-01" """;` * `var filterInput = """Birthday == "2022-07-01T00:00:03" """;` * `var filterInput = """Birthday == "2022-07-01T00:00:03+01:00" """;` + * QueryKit reads a value without an offset, for example `"2022-07-01T00:00:03"`, as UTC. A value with an offset becomes its UTC time. For a `timestamp without time zone` column, see [Date and Time Values Without an Offset](#date-and-time-values-without-an-offset). * `TimeOnly`: * `var filterInput = """Time == "12:30:00" """;` @@ -814,9 +815,31 @@ var config = new QueryKitConfiguration(config => var filterExpression = FilterParser.ParseFilter(input, config); ``` +#### Date and Time Values Without an Offset + +`DateTimeKindForValuesWithoutOffset` sets the `DateTimeKind` of a `DateTime` filter value that has no offset, for example `"2024-01-15T08:00:00"`. The setting applies to a single value, a quoted value, a value in a `^^` list, and a custom operation value. + +| Value | A value without an offset | A value with an offset | Use it for | +|---|---|---|---| +| `DateTimeKind.Utc` (default) | UTC, same wall-clock time | Its UTC time, kind `Utc` | A `timestamp with time zone` (`timestamptz`) column | +| `DateTimeKind.Unspecified` | Same wall-clock time, kind `Unspecified` | Its UTC time, kind `Unspecified` | A `timestamp without time zone` column | +| `DateTimeKind.Local` | The server time zone, kind `Local` | Its server-local time, kind `Local` | The v1.14.2 behavior | + +Npgsql needs a `Utc` value for a `timestamptz` column and rejects a `Utc` value for a `timestamp without time zone` column. The setting applies to every `DateTime` property in the filter. Thus one filter cannot compare a `timestamptz` column and a `timestamp without time zone` column on Npgsql. + +```csharp +var config = new QueryKitConfiguration(config => +{ + config.DateTimeKindForValuesWithoutOffset = DateTimeKind.Unspecified; +}); +var filterExpression = FilterParser.ParseFilter(input, config); +``` + +A `DateTimeOffset` value is always UTC. With `DateTimeKind.Local`, QueryKit reads a `DateTimeOffset` value without an offset in the server time zone and then converts it to UTC. With the other values, QueryKit reads it as UTC. + #### Ignored Clause Behavior -`IgnoredClauseBehavior` and `ParameterizeFilterValues` both come from the `IQueryKitFilterBehavior` interface. `QueryKitConfiguration` implements this interface, so a custom configuration class can implement `IQueryKitFilterBehavior` directly instead. +`IgnoredClauseBehavior`, `ParameterizeFilterValues`, and `DateTimeKindForValuesWithoutOffset` all come from the `IQueryKitFilterBehavior` interface. `QueryKitConfiguration` implements this interface, so a custom configuration class can implement `IQueryKitFilterBehavior` directly instead. `IgnoredClauseBehavior` controls what QueryKit does with a clause on a property that has `PreventFilter`, or on an unknown property when `AllowUnknownProperties` is `true`. The default is `IgnoredClauseBehavior.Remove`, which drops the clause, so a logical operator with a removed side keeps only its other side. Set `IgnoredClauseBehavior` to `IgnoredClauseBehavior.ReplaceWithTrue` for the v1.14.2 behavior. This value replaces the clause with `true == true` so the rest of the expression keeps its shape. With `ReplaceWithTrue`, a property-list clause (for example `(FirstName, Title) @=* "x"`) where every property is prevented gives a plain `true` instead, not `true == true`. diff --git a/SharedTestingHelper/Fakes/FakeTestingPersonBuilder.cs b/SharedTestingHelper/Fakes/FakeTestingPersonBuilder.cs index 70c4a4b..64d908c 100644 --- a/SharedTestingHelper/Fakes/FakeTestingPersonBuilder.cs +++ b/SharedTestingHelper/Fakes/FakeTestingPersonBuilder.cs @@ -9,6 +9,7 @@ public class FakeTestingPersonBuilder .RuleFor(x => x.FirstName, faker => faker.Name.FirstName()) .RuleFor(x => x.LastName, faker => faker.Name.LastName()) .RuleFor(x => x.Title, faker => faker.Lorem.Sentence()) + .RuleFor(x => x.LocalDateTime, _ => null) .Generate(); public FakeTestingPersonBuilder WithTitle(string? title) @@ -71,6 +72,12 @@ public FakeTestingPersonBuilder WithSpecificDateTime(DateTime dateTime) return this; } + public FakeTestingPersonBuilder WithLocalDateTime(DateTime? dateTime) + { + _baseTestingPerson.LocalDateTime = dateTime; + return this; + } + public FakeTestingPersonBuilder WithDate(DateOnly? date) { _baseTestingPerson.Date = date;