fix(docker): Add default crontab download command

[rpatterson]

Clarify the /config/ytdl-sub-configs/cron script with explanatory comments and a default --dry-run command. Also change the wrapper script to echo commands for easier debugging. To update an existing script, move the old script aside, restart the container to regenerate it, and edit the new script.

[rpatterson]

Also clarifies the Automating page in the Getting Started guide docs.

[rpatterson]

Should be merged after #1318. It's not based on it and I'm not aware of any conflicts, but for max narrative consistency, it would be better the merge the earlier one first.

[jmbannon]

while this is helpful for us devs, I dont think the user should see this. Let's remove

[rpatterson]

I added that for users to help them debug their cron script. The user's script is outside the image's control but the wrapper is under it's control and there are user errors where echoed commands could help them understand their mistake. Thoughts?

[jmbannon]

I think users adding their own echo's are sufficient. Seeing echo command followed by the actual echo output seems odd for default behavior with no way to disable

[jmbannon]

ytdl-sub can't run in parallel via file lock checks. Lets omit this wording.

[rpatterson]

Ah, I made a mental note to ask about that at the time. I should not rely on mental notes. ;-)

[rpatterson]

Actually, where is that behavior documented, @jmbannon ? I don't recall coming across it. Happy to find a good place to clarify that.

[jmbannon]

It's sort of hidden from the user, but can kinda be found here: https://ytdl-sub.readthedocs.io/en/latest/config_reference/config_yaml.html#ytdl_sub.config.config_validator.ConfigOptions.lock_directory

[jmbannon]

lets also have cron run on start here with it set to false

[rpatterson]

Again, as a throttle footgun, consider a user who copies this snippet to their Compose config and then after running successfully for months wants to download outside the schedule. We want that user to come back to the docs and read the warning, but if they have that in their config it encourages them to just change it to true and they miss the warning.

[jmbannon]

Im still not convinced its as bad as you think, throttle protection ideally will mitigate most of the issue. Worst case is they skip over their subscription sleep. The other flag already gives them control to start it every 1m so I say give them the rope 😅

[rpatterson]

Im still not convinced its as bad as you think

I was thinking about that after my previous comment, if this is an issue for users it's unlikely to be attributed to this env var. At best, a user will notice they've been throttled during the run triggered by the container start. At worst, they're getting throttled daily when their container is restarted for any of the reasons above, but downloads still happen before a restart happened too close to a scheduled run, so all they notice is "sluggish" downloads of new entries. New entries sometimes download "on-time" but sometimes days "late". Either way, they will approach it as a throttle issue and almost never notice the correlation to container restarts.

throttle protection ideally will mitigate most of the issue. Worst case is they skip over their subscription sleep.

My experience with triggering throttles, and not just resolution throttles, and bans indicates the opposite. My efforts to tune throttle protection have had limited success and even when I was most successful it was extremely sensitive to how frequently I re-ran. That was the other reason I was setting alarms for the end of the sleep_per_download_s: delay, I triggered throttles often when I short cut the download delay even by just 1/3-1/4.

The other flag already gives them control to start it every 1m so I say give them the rope 😅

There's a big difference when guiding new users, and this is the Getting Started guide, between a feature with a safe default that a user can abuse if they figure out how, and a toggle feature.

[jmbannon]

I'm still under the impression it can be toggled in a safe manner. The current representation of it commented out seems like it's a legacy do-not-use feature when it has been commonly requested and used (#730).

I understand the sentiment to protect new users, and I think the warning message below does a good job of that (maybe an addition to the FAQ would too). My friendly ask is to keep the yaml as is:

      environment:
        CRON_SCHEDULE: "0 */6 * * *"
        CRON_RUN_ON_START: false