With our simple Azure Functions web service in place, we can now focus on providing more complex functionality from our service. We will focus on read-only service features for now, since we’re reading game data and no persisting any data yet. We want to build a service that will return lists of elements based on simple search and paging criteria… to support web app operations for filtering, sorting, and pagination. Many apps require this functionality, so it would be good to have a common mechanism to support it.
Also, because we want to investigate various backend Azure storage technologies, it makes sense to create a layer of abstraction between our service logic and the backend data access components. We will implement this abstraction using the Repository pattern, so that the storage logic can change without having to re-write the service logic.
A repository performs the tasks of an intermediary between the domain model layers and data mapping, acting in a similar way to a set of domain objects in memory. Client objects declaratively build queries and send them to the repositories for answers. Conceptually, a repository encapsulates a set of objects stored in the database and operations that can be performed on them, providing a way that is closer to the persistence layer. Repositories, also, support the purpose of separating, clearly and in one direction, the dependency between the work domain and the data allocation or mapping.
Martin Fowler
To implement this pattern, we will:
- Define a Repository interface that our services will use when they want to get objects from our storage backend.
- Implementations of the Repository interface for each storage technology we want to support.
- Generic Repository implementations since the code to retrieve types is the same but the types being used or returned are specific.
- Rich test coverage of the repository logic.
Let’s start by defining the Repository interface.
Repository Interface – IReadableRepository<T, TId>
The first design choice we made was to define an interface for repositories that support read operations (the IReadableRepository interface), which are separate from an interface for edit operations (create, update, delete). Since we will have some repositories that are read-only and others that are read-write (in future lessons), this is a separation that allows us to use the same read operations in either case.
The next design decision was using generics to build the interface and implementation classes, so that they could be used across different input and return types. T is the type of objects operated upon by the repository. TId is the type for the unique identifier of objects in the storage system, which allows us to define identifiers as ints or Guids or some other identifier depending on our requirements. Note: if you need a refresher on C# generics, please read this article.
These design choices were made before we’ve even written a line of code… Let’s take a look at the code to see how this all works.
1. We define a new base class for all of our data objects called NamedElement in the SimpleRPG.Game.Services project and DTO folder.
using System;
namespace SimpleRPG.Game.Services.DTO
{
public abstract class NamedElement<T> : IEquatable<NamedElement<T>>
where T : struct
{
private string name;
protected NamedElement(T id, string name)
{
Id = id;
Name = name;
}
protected NamedElement()
{
}
public T Id { get; set; }
public string Name
{
get => name;
set => name = !string.IsNullOrEmpty(value) ? value : throw new ArgumentNullException(nameof(Name));
}
public override string ToString()
{
return $"{Name} [{Id}]";
}
public bool Equals(NamedElement<T> other)
{
return Id.Equals(other.Id);
}
public override bool Equals(object obj)
{
if (obj is NamedElement<T> element)
{
return Equals(element);
}
return false;
}
public override int GetHashCode()
{
return Id.GetHashCode();
}
}
}
This is a simple abstract data class that all of our top-level repository data transfer objects will derive from. It ensures that we have a minimal structure for a data entity of Id and Name. As we can see the Id property uses a generic definition, so the identifier can be any reference type. Line #6 constrains the types than can be used for this class.
This class also implements the IEquatable interface, so that we can implement custom equality operations. In this code, any elements that have equal Id values are considered equal. The default implementation is reference equality, which we want to override. As we can see, we override a couple of Equals and GetHashCode methods to ensure they all follow the same behavior.
This is a simple type, but enables some powerful capabilities in equality and in our repository later.
2. Derive ItemTemplate from this new base.
namespace SimpleRPG.Game.Services.DTO
{
public class ItemTemplate : NamedElement<int>
{
public int Category { get; set; }
public int Price { get; set; }
public string Damage { get; set; } = string.Empty;
public int Heals { get; set; }
}
}
Initially, our ItemTemplate class had its own definition of Id and Name properties, so we modified the class to derive from NamedElement with a type of int to match our existing definition. This preserves the properties we had and gives us the NamedElement implementation.
3. Define a NameValuePair data class in the DTO folder. We are going to use property bags to pass some unstructured filter and sorting data between our service and repository, so the NameValuePair class helps us do that.
using System;
namespace SimpleRPG.Game.Services.DTO
{
public class NameValuePair<TValue> : IEquatable<string>
{
private string name;
public NameValuePair(string name, TValue value)
{
Name = name;
Value = value;
}
public NameValuePair()
{
}
public string Name
{
get => name;
set => name = !string.IsNullOrEmpty(value) ? value : throw new ArgumentNullException(nameof(Name));
}
public TValue Value { get; set; }
public override string ToString()
{
return $"[{Name}] = {Value}";
}
public bool Equals(string other)
{
return Name.Equals(other, StringComparison.InvariantCulture);
}
public override bool Equals(object obj)
{
if (obj == null)
{
return false;
}
var other = (NameValuePair<TValue>)obj;
return Equals(other.Name);
}
public override int GetHashCode()
{
return Name.GetHashCode();
}
}
}
This defines a pair with a string name and any type as its value. This is similar in structure to elements in a Dictionary, but is a public type that we can use in multiple places. We could have also used the .NET Core Tuple class for this as well (since the structure isn’t extremely important), but I prefer to use a specific type for clarity of purpose.
This class also implements the IEquatable interface to test for equal objects by their Name property.
4. Define the IReadableRepository interface in the SimpleRPG.Game.Services project and Repositories folder.
using SimpleRPG.Game.Services.DTO;
using System.Collections.Generic;
using System.Threading.Tasks;
namespace SimpleRPG.Game.Services.Repositories
{
public interface IReadableRepository<T, TId>
where T : NamedElement<TId>
where TId : struct
{
Task<IEnumerable<T>> GetEntities(
int? offset,
int? limit,
IEnumerable<NameValuePair<string>> filters,
IEnumerable<NameValuePair<string>> sorts);
Task<int> GetEntityCount(IEnumerable<NameValuePair<string>> filters);
Task<T> GetEntityById(TId id);
Task<IEnumerable<T>> GetBulkEntitiesById(IEnumerable<TId> ids);
}
}
Let’s look at each section of this interface definition:
- Line #7, the interface is defined with 2 generic parameters. The first is for the type we retrieve via the read operations. The second is the type of the entity’s
Idproperty, which allows us to support multiple forms of identity – i.e.intandGuid. - Line #8, constrains the element type to only allow classes derived from
NamedElement<TId>. Therefore all of our top-level data transfer objects returned from our repositories must derive fromNamedElement. - Line #9, constrains the id type to be a value type.
- All of the repository methods are async, so they have a
Task<T>return type. Because these methods are returning data that can be retrieved from external data sources, we want the operations to be asynchronous so they don’t block the processing of our Azure Functions. - The
GetEntitiesmethod (line #11-15) retrieves a list of elements. If it’s called with all null parameters, then it would return the list of all entities in the repository. Many times repositories have an upper limit to prevent large query results from degrading performance.- offset is in pagination to set the starting position of the retrieval cursor. For example if we wanted the start of the third page (with 10 items per page), the value would be 21.
- limit is also used for pagination or to limit the total result returned. For example if we wanted to get the top 10 results, the value would be 10.
- filters is a list of name-value pairs that represents simple filters we apply to our query. These are typically filters that users can specify in the user interface.
- sorts is also a list of name-value pairs that represents how the return data should be sorted (which properties to sort by).
- This method definition is a bit complex, but gives great flexibility to our web service API. For simple queries, these filters and sorts can be used. If we have a custom query with complex logic and requirements, then we could be a specific
GetMyComplexDatamethod, but that would be an extension to theIReadableRespositoryinterface. - The
GetEntityCountmethod (line #17) returns the count of the list represented by the filters. This call is used many times in conjunction with pagination and multipleGetEntitiesmethods to build that functionality. GetEntityByIdmethod (line #19) retrieves a single entity by itsIdproperty. The search parameter is of typeTId. There should only be a single item with a given id.- Finally, the
GetBulkEntitiesByIdmethod (line #21) takes a list of ids and returns the list of corresponding entities. The entities are returned in the order of requested ids. This could be useful in returning the list of item information that our player is carrying… all in one request, without having to do a query for each item.
This is a very general purpose interface that can be used for all of our game data repositories. We just have to specify the data transfer type that we are intending to retrieve. This repository interface may also be useful in other web service projects… I typically define an interface like this in a common library for all of my services to share.
Build InMemoryReadRepository Class
With our interface and types defined, we need an implementation class to make use of interface. Since so much of our code can be shared between repository implementations, we are going to build a base class to manage in-memory repositories. These are lists kept in memory (or easily loaded) and manipulated by our web services – this is how we will use the repository pattern for our embedded resource data files.
The InMemoryReadRepository class is also a generic class that implements the IReadableRepository interface. That way consuming code only communicates via the IReadableRepository interface and doesn’t know which repository implementation is actually being used. This layer of abstraction makes our service logic portable to different storage technologies.
We’ll start with the full source listing in the SimpleRPG.Game.Services project and Repositories folder, and then review each method for what it does.
using SimpleRPG.Game.Services.DTO;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
namespace SimpleRPG.Game.Services.Repositories
{
public class InMemoryReadRepository<T, TId> : IReadableRepository<T, TId>
where T : NamedElement<TId>
where TId : struct
{
private readonly Dictionary<string, Func<T, string, bool>> filterChecks = new Dictionary<string, Func<T, string, bool>>();
private readonly Dictionary<string, Func<T, object>> sortChecks = new Dictionary<string, Func<T, object>>();
public InMemoryReadRepository(
IList<T> entities,
IList<NameValuePair<Func<T, string, bool>>> knownFilters = null,
IList<NameValuePair<Func<T, object>>> knownSorts = null)
: this()
{
Entities = entities ?? throw new ArgumentNullException(nameof(entities));
if (knownFilters != null)
{
foreach (var f in knownFilters)
{
filterChecks.Add(f.Name, f.Value);
}
}
if (knownSorts != null)
{
foreach (var s in knownSorts)
{
sortChecks.Add(s.Name, s.Value);
}
}
}
public InMemoryReadRepository()
{
filterChecks.Add("Name", (f, v) => f.Name.Contains(v, StringComparison.InvariantCultureIgnoreCase));
sortChecks.Add("Name", p => p.Name);
sortChecks.Add("Id", p => p.Id);
}
protected virtual IList<T> Entities { get; set; } = new List<T>();
public Task<IEnumerable<T>> GetEntities(
int? offset,
int? limit,
IEnumerable<NameValuePair<string>> filters,
IEnumerable<NameValuePair<string>> sorts)
{
var o = offset ?? 0;
var l = limit ?? Entities.Count;
var filtered = ApplyFilters(Entities, filters);
var sorted = GetSortedEntities(filtered, sorts);
return Task.FromResult(sorted.Skip(o).Take(l).ToList().AsEnumerable());
}
public Task<int> GetEntityCount(IEnumerable<NameValuePair<string>> filters)
{
var filtered = ApplyFilters(Entities, filters);
return Task.FromResult(filtered.Count());
}
public Task<T> GetEntityById(TId id)
{
var spell = Entities.FirstOrDefault(t => t.Id.Equals(id));
if (spell == null)
{
throw new EntityNotFoundException(nameof(id), id);
}
return Task.FromResult(spell);
}
public Task<IEnumerable<T>> GetBulkEntitiesById(IEnumerable<TId> ids)
{
_ = ids ?? throw new ArgumentNullException(nameof(ids));
var entities = (List<T>)Entities;
var list = entities.FindAll(p => ids.Contains(p.Id));
return Task.FromResult(list.OrderBy(e => e.Name).ToList().AsEnumerable());
}
protected virtual IEnumerable<T> ApplyFilters(
IEnumerable<T> items,
IEnumerable<NameValuePair<string>> filters)
{
if (filters == null)
{
return items;
}
foreach(var f in filters)
{
if (filterChecks.TryGetValue(f.Name, out var filterAction))
{
items = items.Where(p => filterAction.Invoke(p, f.Value)).ToList();
}
}
return items;
}
protected virtual IEnumerable<T> GetSortedEntities(
IEnumerable<T> items,
IEnumerable<NameValuePair<string>> sorts)
{
if (sorts == null)
{
if (sortChecks.TryGetValue("Name", out var sortAction))
{
items = items.OrderBy(p => sortAction.Invoke(p)).ToList();
}
return items;
}
foreach (var s in sorts)
{
if (sortChecks.TryGetValue(s.Name, out var sortAction))
{
if (s.Value.ToLower() == "dsc")
{
items = items.OrderByDescending(p => sortAction.Invoke(p)).ToList();
}
else
{
items = items.OrderBy(p => sortAction.Invoke(p)).ToList();
}
}
}
return items;
}
}
}
- Right from the class declaration (lines 39-11), we see that
InMemoryReadRepositoryis a generic class with the same types and constraints as theIReadableRepositoryinterface. The class implements the methods required by the interface too. - Then we declare two member variables (lines # 13-14), which are dictionaries that hold functions responsible to filtering and sorting our types. These functions contain the type specific logic for corresponding Linq operations. These are anonymous functions that will be called later. We will describe their use in later methods.
- We provide 2 constructors:
- One initializes the
Entitieslist and the filtering/sorting functions. This allows derived types and factories to provide type specific checks. - Also the default constructor which starts with an empty
Entitieslist. Then it adds filter logic for filtering by elementName, and sorting functions forIdandName. (Since we are working withNamedElementderived classes, we know that they all have Id andNameproperties).
- One initializes the
- The protected
Entitiesproperty (line #49) is only accessible to this or derived classes. It holds our list in memory and operations are performed upon it. GetEntitiesmethod (lines # 51-63) orchestrates the steps to querying for the entities:- first it ensures values for offset and limit, even though they are optional (by using defaults).
- then calls the
ApplyFiltersmethod over the whole list to get a filtered list. - then calls the
GetSortedEntitiesmethod over the filtered list. - finally it returns the Linq query of the filtered list with the appropriate offset and limit.
- because this method is async while all of its internal code is synchronous, we return a
Taskwith our result. When we use asynchronous storage APIs (for Azure Store or CosmosDB), the async methods will make more sense.
GetEntityCountmethod (lines #65-69) just applies the specified filters (via the sameApplyFiltersmethod) to the fullEntitieslist and then returns its count. We only apply filters in this code path because sorting doesn’t affect the number of entities returned.GetEntityByIdmethod (lines #71-80) uses the specified id parameter to search theEntitieslist for that item. It uses theFirstoperation, so will only return the first, single element with that id, but they should be unique. If the id specified cannot be found in this repository, we throw anEntityNotFoundException(we will describe that exception class later).GetBulkEntitiesByIdmethod (lines #82-89) takes a list of ids and finds the list of Entities that they represent. This uses the LinqFindAlloperation to find the set that match.ApplyFiltersmethod (lines #91-109) does the heavy lifting of finding and applying filters using the LinqWhereoperation.- If no filters were applied, the method just returns the full list.
- For every filter that was passed into the call, it looks up the filter function in the
filterChecksdictionary using itsName. - If that filter check is found in the dictionary, then we add a
Whereclause and invoke the filter function from that dictionary as the predicate… thereby reducing our return list.Whereoperations will be chained for multiple filters. - If a filter is passed to this method but it is not found in the
filterChecksdictionary, then that filter operation is skipped.
GetSortedEntitiesmethod (lines # 111-141), similar toApplyFilters, performs the logic of sorting the specified list of entities.- First, if there is no sorting specified, then the default behavior is to sort by
Name. Without any sort applied the elements would return in their order in the list, but that’s not usually a very intuitive order, so sorting by name is a reasonable default. - For every sort entry that was passed into the call, it looks up the sort function in the
sortChecksdictionary using itsName. - If that sort check is found in the dictionary, then we add an
OrderByorOrderByDescendingclause and invoke the sort function from that dictionary as the predicate… thereby sorting our return list.OrderByandOrderByDescendingoperations will be chained for multiple sort entries. - The sort entry
Valuecan either specify asc (ascending) or dsc (descending) sort order, but it defaults to ascending (if dsc is not specifically used). - If a sort entry is passed to this method but it is not found in the
sortChecksdictionary, then that sort operation is skipped.
- First, if there is no sorting specified, then the default behavior is to sort by
This is a very generalized class with a lot of code, but it performs some powerful operations on in-memory data lists. Because of its general implementation, it can and will be used as the base class for our in-memory repositories in future lessons.
Finally, let’s look at the simple EntityNotFoundException class in the Repositories folder. We use a specific exception for this rather than the general .NET exception that could have been sent, so that we can take actions on it in our service.
using System;
namespace SimpleRPG.Game.Services.Repositories
{
public class EntityNotFoundException : Exception
{
public EntityNotFoundException(string entityIdName, object entityIdValue, Exception innerException = null)
: base($"Entity with {entityIdName} = {entityIdValue} was not found in repository.", innerException)
{
this.EntityIdName = entityIdName;
this.EntityIdValue = entityIdValue;
}
public string EntityIdName { get; private set; }
public object EntityIdValue { get; private set; }
}
}
This repository implementation is the type of code that I like to implement once and use everywhere, so it is also a good candidate for code in a common library. The beauty of sharing this code is that we can fully test the behavior with all of the cases we need to support and then just use them in all of our service projects without having to worry about the implementation details.
Testing the InMemoryReadRepository
To verify the functionality of the in-memory repository, we created a set of tests that go through the various capabilities described above and ensure that the InMemoryReadRepository class meets all of those requirements. We’re not going to exhaustively describe all of the tests, but just how to get the testing mocks and infrastructure in place with a few example tests.
We’ll start with a data class that derives from NamedElement called TestElement.
using SimpleRPG.Game.Services.DTO;
using System;
namespace SimpleRPG.Game.Services.Tests.Mocks
{
public class TestEntity : NamedElement<int>
{
public int Value { get; set; }
public DateTime ModifiedDate { get; set; }
public string Author { get; set; }
}
}
Then, we define a derived repository called MemoryTestRepository. It derives from InMemoryReadRepository and just configures it with the right starting data (5 test entities), known filters for Value and Author, and known sorting for Value and Author.
using SimpleRPG.Game.Services.DTO;
using SimpleRPG.Game.Services.Repositories;
using System;
using System.Collections.Generic;
namespace SimpleRPG.Game.Services.Tests.Mocks
{
public class MemoryTestRepository : InMemoryReadRepository<TestEntity, int>
{
private static readonly List<TestEntity> AllElements = new List<TestEntity>
{
new TestEntity
{
Id = 1,
Name = "Element1",
Value = 42,
ModifiedDate = DateTime.Today,
Author = "DarthPedro",
},
new TestEntity
{
Id = 2,
Name = "Element2",
Value = 3,
ModifiedDate = DateTime.Today,
Author = "DarthPedro",
},
new TestEntity
{
Id = 33,
Name = "Element3",
Value = 17,
ModifiedDate = DateTime.Today,
Author = "DarthPedro",
},
new TestEntity
{
Id = 4,
Name = "Element4",
Value = 45,
ModifiedDate = DateTime.Today,
Author = "DarthPedro",
},
new TestEntity
{
Id = 5,
Name = "Element5",
Value = 46,
ModifiedDate = DateTime.Today,
Author = "DarthPedro-2",
},
};
private static readonly List<NameValuePair<Func<TestEntity, string, bool>>> knownFilters =
new List<NameValuePair<Func<TestEntity, string, bool>>>
{
new NameValuePair<Func<TestEntity, string, bool>>("Value", (f, v) => f.Value == Convert.ToInt32(v)),
new NameValuePair<Func<TestEntity, string, bool>>("Author", (f, v) => f.Author.Contains(v, StringComparison.InvariantCultureIgnoreCase)),
};
private static readonly List<NameValuePair<Func<TestEntity, object>>> knownSorts =
new List<NameValuePair<Func<TestEntity, object>>>
{
new NameValuePair<Func<TestEntity, object>>("Value", p => Convert.ToInt32(p.Value)),
new NameValuePair<Func<TestEntity, object>>("Author", p => p.Author)
};
public MemoryTestRepository(List<TestEntity> initialElements)
: base(initialElements)
{
}
public MemoryTestRepository()
: base(new List<TestEntity>(AllElements), knownFilters, knownSorts)
{
}
}
}
Finally let’s look at a couple of tests to see how these mock classes are used, and how we validate the InMemoryReadRepository functionality.
using SimpleRPG.Game.Services.DTO;
using SimpleRPG.Game.Services.Repositories;
using SimpleRPG.Game.Services.Tests.Mocks;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Xunit;
namespace SimpleRPG.Game.Services.Tests.Repositories
{
public class InMemoryReadRepositoryTests
{
...
[Fact]
public async Task GetEntityById()
{
// arrange
var id = 2;
var repo = new MemoryTestRepository();
// act
var result = await repo.GetEntityById(id).ConfigureAwait(false);
// assert
Assert.NotNull(result);
Assert.Equal(id, result.Id);
Assert.Equal("Element2", result.Name);
Assert.Equal(3, result.Value);
Assert.Equal(DateTime.Today, result.ModifiedDate);
Assert.Equal("DarthPedro", result.Author);
}
[Fact]
public async Task GetEntities_FirstPage()
{
// arrange
var repo = new MemoryTestRepository();
// act
var spells = await repo.GetEntities(0, 2, null, null).ConfigureAwait(false);
// assert
Assert.Equal(2, spells.Count());
Assert.Equal(1, spells.First().Id);
Assert.Equal(2, spells.Last().Id);
}
[Fact]
public async Task GetEntities_FilteredByMultiple()
{
// arrange
var repo = new MemoryTestRepository();
var filters = new List<NameValuePair<string>>
{
new NameValuePair<string>("Name", "element"),
new NameValuePair<string>("Author", "DarthPedro-2"),
};
// act
var spells = await repo.GetEntities(null, null, filters, null).ConfigureAwait(false);
var count = await repo.GetEntityCount(filters);
// assert
Assert.NotEmpty(spells);
Assert.Equal(1, count);
Assert.Equal(5, spells.First().Id);
}
...
}
}
These 3 tests show the structure of the rest. They show how to search for an item by id, get a paged list of items, and filtering items to a sub-list. As we can see the code for retrieving data is very simple – much of its complexity is contained in the InMemoryRepository. The setup code in these tests are also a great blueprint for the code we will need to write in our service layer to integrate with this repository.
If you wish to see the comprehensive test suite for this repository, please review the commit associated with this lesson. It contains test for all of the the scenarios described in this lesson.
In conclusion, we’ve built a powerful abstraction for our data sources using the Repository pattern. We defined a generic interface and class for the repository. And we built tests that show how the repository can be used. In the next lesson, we will update the ItemTemplateService to retrieve its data from a repository rather than directly from the resource file.
DarthPedro's Blog 