Implement a max_retries option

[Nihlus]

This PR adds the ability for rebuilders set an upper limit for the number of times BAD packages will be retried.

Currently, bad packages are retried indefinitely, taking up queue space and valuable server time. In most cases, a BAD build will never become GOOD, and as such it's been an oft-requested feature to turn that indefinite retry off.

With these changes, administrators can configure the maximum number of allowed retries on packages in the configuration file. It does not presently distinguish between retries on BAD vs FAIL packages, but that is an enhancement we can make in the future.

[kpcyrd]

I think ends the entire loop for all packages, instead of proceeding with the next package. :)

To avoid this, it could be written like:

if let Some(max_retries) = cfg.schedule.max_retries()
        && retry_count >= max_retries as i32 {
    // null out the next retry to mark the package as non-retried
    update(build_inputs::table)
        .filter(build_inputs::id.eq(build_input.id))
        .set(build_inputs::next_retry.eq(None::<NaiveDateTime>))
        .execute(conn)?;

    // drop any enqueued jobs for the build input
    delete(queue::table)
        .filter(queue::build_input_id.eq(build_input.id))
        .execute(conn)?;
} else {
    let priority = match current_status {
        BuildStatus::Bad => Priority::retry(),
        _ => Priority::default(),
    };
    let new_queued_job = NewQueued {
        build_input_id: build_input.id,
        priority,
        queued_at: Utc::now().naive_utc(),
    };
    new_queued_job.upsert(conn.as_mut())?;
}

This is using the if-let-chain feature, we may have to change the Rust edition to 2024 for this feature (which shouldn't break anything though). Already done on the current git main. :)

[jspricke]

This is deployed on reproduce.debian.net but does not seem to work. I just saw on arch:all haskell-config-schema being built and afterwards queued again even though it has been tried 23 times already.