Decrease RAM requirements

[glen3b]

Score data is very redundant in RAM. We should work on reducing the bot's RAM usage through various means, including improving this.

[glen3b]

Possibility: use a relational database

[glen3b]

Another possible consideration: binary serialization and caching.

A relational database makes a lot of sense. I'll have to rework how I do "expensive" queries but that's OK.

[glen3b]

Thoughts on relational database:

• Remove the "cheap query" score provider metadata property in favor of...
• Adding these two functions

        Task<IEnumerable<TResult>> QuerySummaryAsync<TResult>(Func<System.Linq.IQueryable<ScoreboardSummaryEntry>, System.Linq.IQueryable<TResult>> executeQuery);
        Task<IEnumerable<TResult>> QueryDetailsAsync<TResult>(Func<System.Linq.IQueryable<ScoreboardDetails>, System.Linq.IQueryable<TResult>> executeQuery);

This way we can hopefully get the benefits of an IQueryable against a database while abstracting away the async call (which is not in the BCL but is ORM-dependent), and this should work against our existing score provider infrastructure. Assuming this actually works, the biggest issue I can think of is handling aggregate queries. I think the async aggregate calls in EF Core are in EF Core (i.e. not BCL), and I'd rather not have this interface dependent on EF Core. This leaves a few options:

• We could add IScoreRetrievalService interface methods separately for aggregates, e.g.

        Task<int> QueryDetailsMeanAsync<TResult>(Func<System.Linq.IQueryable<ScoreboardDetails>, System.Linq.IQueryable<int>> selectorQuery);
        Task<int> QueryDetailsSumAsync<TResult>(Func<System.Linq.IQueryable<ScoreboardDetails>, System.Linq.IQueryable<int>> selectorQuery);
        Task<double> QueryDetailsMeanAsync<TResult>(Func<System.Linq.IQueryable<ScoreboardDetails>, System.Linq.IQueryable<double>> selectorQuery);
        Task<double> QueryDetailsSumAsync<TResult>(Func<System.Linq.IQueryable<ScoreboardDetails>, System.Linq.IQueryable<double>> selectorQuery);
        // etc

I think this would get out of hand.

• Let the aggregates evaluate C#-side. It certainly avoids a hassle, for some cases (e.g. histograms) we have to do this anyway, and at our scale I don't think the performance impact would be that great.
• Add some interface for aggregate queries that we expose through the library and implement on the database provider side.