[Dynamic Selection] Customization of Backends and Policies

[danhoeflinger]

Dynamic Selection API: Backend and Policy Customization & Removal of Selection API

This PR refactors the Dynamic Selection API to introduce a flexible backend architecture and simplify the user-facing API by removing the select() function. It provides better tools for customization of backend and policies to allow for easier customization and more flexibility for users.

Implements RFCs #2220.

Key Changes

Backend Architecture

• Added policy_base.h and default_backend.h to provide common base classes for policies and backends respectively
• Backends now accept ResourceAdapter function to support different flavors of resource with the same backend (e.g. sycl::queue vs sycl::queue*)

API Simplification

• Removed select() function - selections are now internal implementation details
• Addition of try_submit- always returns a std::optional quickly, returns empty if unable to obtain a resource
• Users now exclusively use try_submit, submit() and submit_and_wait() functions
• Policies expose try_select_impl() (returns std::optional) instead of public select()

Execution Info & Reporting

• Backends validate reporting requirements at construction and filter incompatible resources
• Clear runtime errors when no compatible resources remain after filtering

---

Summary of changes to look out for from individual components:

• Documentation:
  • Removal of public select API
• Examples:
  • Policy template argument adjustment (by resource first rather than backend)
• Policies:
  • Adjustment to use policy_base, remove repetative code
  • Implement try_select_impl instead of select
  • Implement initialize_impl instead of initialize
• Sycl Backend:
  • Conversion to partial specialization of default backend, use of backend_base
  • Addition of ResourceAdapter to support different flavors from a single base resource / backend
  • Improve system of reporting requirements for better filtering and error messages
  • Adjust support for profiling to require sycl extension for profiling (only reliable way to time)
    • Filter devices accordingly
• Tests:
  • Removal of testing of select before submit
  • Addition of default universe initialization testing
  • Addition of testing of use of resource adapter to support sycl::queue* in addition to sycl::queue

[SergeyKopienko]

I think we can avoid this loop by using https://en.cppreference.com/w/cpp/container/vector/insert.html

[danhoeflinger]

@egfefey Can you take a look at this?

I responded before on this topic that we would rather not expand the scope of this PR further as this is the way this is generally done in dynamic selection to this point. I'm not sure if this causes any problems or presents and limitations on what can be used for resources.

[egfefey]

We didn't purposefully do any push_back/insert comparison but considering the fact that we only deal with a handful of resources, any possible impact is likely to be minimal. In any case, I will create a github issue so we can collect performance numbers later to justify a switch if the need be.

[SergeyKopienko]

I think it's the kind of approach which just better to fix: std::vector::insert exactly allows to us avoid these loops.
Also from time to time memory allocations will happens on this code and I think in case of insert at first we will have allocation -> copy -> insert new and in your current approach we will have insert new -> allocation -> copy (existed earlier + already inserted) -> insert new (the rest).

So my opinion - I still think that better to fix in this PR because it's the new code.

[SergeyKopienko]

The same; may be we can use https://en.cppreference.com/w/cpp/container/vector/insert.html ?

[danhoeflinger]

@egfefey this is related to above.

[egfefey]

Same response here.