v1.8 ~ audio offset?! 1000Hz pollrate?!?! no more "magic bpm"! ~

Back to "real" changes this time, with two BIG features and a couple extras.
Only one big hurdle left to complete my long yearned for pop'n dream (maybe for v2.0 ? 👀 )

Special thanks

Huge thanks to Roganis, pulse, though, xinrin, Milo and babyzionov for discussing and/or beta-testing the new big features.
I think that without them these new features would have been released in a much less refined state, so once again thank you very much :D

Thanks to r2nk226 whose practice mode patch taught me how to display info on screen, which proved VERY useful for this version.

Thanks to leonid, fishluv, though and pulse who shared a great deal of information about magic bpm on various discords along the years :D

New additions

<disable_keysounds>

You can now disable keysounds. They will be played automatically at the right time.

This actually makes the game rewrite the whole chart on-the-fly when it's loaded, to convert it into a non-keysounded one, how cool is that!

Note: extra keysounds tied to buttons after the end of the song are still there (like Kenshiro screaming "atatatatatatata" at the end of Hokuto no Ken opening theme, or "Je t'aime Oscar!" at the end of Versailles no Bara), I wouldn't want to take away the fun from these charts :D

<audio_offset>

This allows to compensate the game audio latency (especially needed for stock cabs which can't benefit from win10 low-latency audio driver, or some DAC setups, etc..).

This patch is the only reason why I worked on disabling keysounds :D

If there are no keysounds, then judgement timing offset (aka "keysound_offset" option) is equivalent to an actual audio offset.

This option is added for convenience, so that you don't have to set the other two options separately.

Note: you cannot use audio_offset if keysound_offset is already set.

<enhanced_polling> (decoupled 1000Hz polling)

Pop'n engine judges your timing based on the gap in milliseconds between your button press and the "expected time", however it checks your inputs only once every 16ms.

This means it cannot accurately track your timing, and what should be a 40ms timing window ( (-22ms <= gap <= +18ms) <=> cool ) becomes a "sometimes 2, sometimes 3 frames" (40/16.6666 = 2.4 frames at 60Hz).

In other words, "40ms timing window" is "sometimes 33.33333ms, sometimes 50ms" instead.

This leads to a number of issues which have been noticed and discussed over the years, like the infamous "magic BPM".

You can skip the next subsection if you're already familiar with these issues : long story short this patch solves them :D

Magic BPM?

This was first observed as "higher score on specific songs when playing on home setup vs playing on cab".

This is a game engine quirk where if you run the game at 60Hz, your score becomes higher than it should be on songs with base BPM 100, 150, 180, or 200.

Indeed, running the game at 60Hz (instead of 59.9Hz like a CRT cab would), for these songs the "sometimes 2, sometimes 3 frames" becomes a "mostly 3 frames"
(it's almost like having a 50ms cool window instead of a badly implemented 40ms one :D)

Further studies have shown that there are in fact issues with any chart whose BPM divides 3600 (and this number has a lot of divisors). See the appendix for more info.

The converse is also theoretically true, some songs at 59.9Hz probably have more 3-frame windows than they should...
(a 3-frame window is always too long while a 2-frame window is always too short anyways)

How it is solved

This patch setups a separate thread which polls your inputs every ms and keeps track of when each button was first pressed, which allows to readjust the judgement to what it should be.

Without the patch:

  13ms too late, you've missed the cool window by 5ms, 
  you have a "great"

With the patch:

  "..., oh but you've been pressing the button for 10ms already 
  => you're really only 3ms too late, you're within the cool window, 
  you have a cool"

TL;DR: This makes the game scoring more accurate (entirely millisecond-based timing), and most importantly, independent of the framerate you're running the game at.

Note: You might find some songs become harder to score on, especially if you were running at 60Hz before, but I assure you this new scoring is "correct" 😅

<enhanced_polling_priority>

You can set a custom thread priority for the polling. This is an underlying call to SetThreadPriority so you can check Win32 API documentation for more info about the different possible values.

I personally have to use "1" on my SD BemaniPC hardware else it wouldn't be able to keep up with 1000Hz polling rate.

Setting it to a higher value had a weird effect on official hardware, where it seemed like the timing judgement function was running a bit late (I would get frequent offset corrections above 16ms... while it is not really a problem since we're correcting it appropriately, I think it should not happen :p)

<enhanced_polling_stats>

You can enable polling stats display to ensure the system is working properly, and maybe find which thread priority is best suited for your system.

Colors have been chosen so that it's readable over the game lanes during song, but if you have trouble, the stats are also positionned in a way that "Sudden+" option will give you a nice black background for them :D

Note: It is normal not to see offset correction values updating while in menus. This only happens during songs.

<debounce >

Since we are running our own input management with enhanced_polling (and since I had to disable calls to usbPadReadLast() on real IO for it to work on cabs), I added a "debounce on button release" which you can set to your needs (I suggest keeping the default value of 5ms, you can bump it a little if you encounter unwanted double-taps).

I'm only debouncing on release because the game doesn't have strict release timing, and this allows to keep a quick reaction to new inputs.

Note: this requires enhanced_polling.

Note for cab owners: I've thoroughly checked and usbPadReadLast() serves no other purpose than debouncing inputs, so we're not losing anything by disabling these calls and rolling our own.

<back_to_song_select>

This option allows you to press numpad 9 on option select (before or after pressing the yellow buttons for settings), in order to go back to song selection. This is useful if you picked that song by mistake (or have second thoughts after seeing the hellish solfan surprise :D)

This causes minor issues with song select menu sounds, which I won't bother fixing, so I didn't integrate it in quick_retire and instead put this separately in the "experimental patches" category. Thanks for your understanding.

Note: when you're back to song select, the cursor is on the last played song rather than the one you just picked. This might get fixed in a later version as ideally I'd like to fix other cursor issues with omni songs.

enhancements

Aside from a bit of code cleanup, here are the notable enhancements.

quick retry

It is now possible to trigger a retry from the result screen.

This will send your current score and bring you back to option select for the same song (which means you can hold 8 to skip option select as well if you want to retry with same options).

Note: Because numpad 7 is used on result screen for "add to favorites", quick retry has been moved to numpad 8 instead. This also applies to quick retry during song and option select skip.

force unlock

force_unlock_deco is gone. The new engine causes less trouble when a string is not found, so there's no need to separate the options anymore, force_unlock will always attempt to unlock deco as well if it can.

xml rewrite

With the number of options growing, popnhax.xml was becoming hard to read. Groupings and rearrangements have been made, I hope it will give users an easier time browsing this file.

APPENDIX: Magic BPM data summary (courtesy of fishluv)

BPMs of 100, 150, 180 and 200 are "magic bpm" because any 16th note in them would get a consistent 3 frame cool timing window.
But in reality, there are also other bpms where it's not the 16ths but rather other note speeds which benefit from this lenient cool timing :

Note speeds equivalent to 16ths at 100, 150, 180, and 200.

64ths at  25 ---  45  50

48ths at ---  50  60 ---

32nds at  50  75  90 100

24ths at --- 100 120 ---

16ths at 100 150 180 200

12ths at --- 200 240 ---

8ths at  200 300 360 400

6ths at  --- 400 480 ---

4ths at  400 600 720 800

effective list of bugged bpms = 25, 45, 50, 60, 75, 90, 100, 120, 150, 180, 200, 240, 300, 360, 400, 480, 600, 720, 800

In conclusion, any BPM which is a divisor of 3600 is "magic".

v1.7 ~ translation and logging, new engine ~

This new release is mostly "QoL" changes. This felt like a lot of work for little visible difference, but I like to think this is some ground work that needed to be done to prepare for future patches and community additions :)
stay tuned!

Also from now on, popnhax releases will include ifs_layeredFS files just in case, since they are required for patch_db and translations...

new additions

translation

popnhax can now apply .ips translation patches or multiple sjis string inline replacements. There is quite a lot info about it so I'll leave that as an appendix.

This behavior can be disabled with disable_translation debug option.

logging

This was long overdue but popnhax will now generate a popnhax.log file in your data folder, which should make troubleshooting easier

enhancements

multiboot setups

you can now use --no-omni commandline arg to force set patch_db to 0 without having to edit your .xml, in case you want a multiboot setup with/without omni

patch_db is also automatically disabled for datecodes < usaneko, and a bugfix regarding the score challenge option has been pushed

new engine

Originally, popnhax came with a fuzzy_search engine to look for offsets within the dll. I already had to fix a couple bugs in it, and while working on translation I noticed a couple instances of unstable behavior, therefore I changed the engine for a simpler Boyer-Moore algorithm (adapted from code found here )

The added good news is it seems to actually be quite faster :D

We didn't actually make use of the "fuzzy" capabilities of the previous engine in popnhax so I think nothing important was lost in the process.

APPENDIX: translation details

If you used translation packs before you probably remember it was comprised of:

• new assets (bunch of .ifs files to replace in data/tex/systemxx/)
• a patched dll (or a .ips patch to generate the patched dll from a clean one).

If the patch is just made of inplace translation of sjis strings, you can now use a .dict file
If it's more elaborated, you can always use a .ips file

Note: both methods are supported at once... the .ips patch will be applied first, then if a .dict file exists, popnhax will attempt to perform the string replacements. So if you spot untranslated strings in your pack (like a ずっと ;) ), you can always try to make a popn22.dict file in addition, and it will probably work :D

.dict file

a .dict file is a sjis encoded (NOT UTF-8!) text file following this format:

;種類順;Type
;名前順;Name
;入手順;Rarity
;戻る;Return
;戻る;Return
;ランク順;Score
;レベル順;Level
;名前順;Name
;ジャンル;Genre
;スコアチャレンジ;Challenge
;ポップ君の落下速度変化
%s;POP KUNS FALLING SPEED
%s
;判定ラインをずらします;SHIFT JUDGEMENT LINE
;判定ラインが定位置です;USUAL JUDGEMENT LINE
;ポップ君を叩いた時の
判定に併せて音が鳴るよ;POP KUNS WILL
MAKE A SOUND.

popnhax will automatically look for the japanese string in the dll, and replace with the corresponding one (note that multiline strings are supported).
If the translated string is too long, it will be truncated automatically and you'll get a warning popping up in logs.

.ips file

Older translation packs sometimes did more than inplace string replacements, and had a .ips file which could feature dll expansion etc... this is also supported and applied/adapted on-the-fly (oh did eclale give me some trouble >__<)

A .ips file is just a list of offsets and data to write at these offsets, so they only work on the exact dll they were intended for. There's no way for popnhax to know if the file you're patching is indeed the intended one, so make sure to check beforehand.

translated assets

Just like any other .ifs mod, the translated assets can just be put in a subfolder of data_mods, thanks ifs_layeredFS as usual

you can now put a _yyyymmddrr_tr folder in data_mods (e.g. _2016120400_tr ), containing the popn22.ips file, as well as the tex subfolder with translated assets, and popnhax will apply it.

How to use: Example

If I want to use eclale with its translation pack, I put the following in data_mods

data_mods/_2016120400_tr/popn22.ips
data_mods/_2016120400_tr/popn22.dict
data_mods/_2016120400_tr/tex/system23/b_gmmain.ifs
data_mods/_2016120400_tr/tex/system23/... [all the other .ifs from the pack]

then I run popn22_2016120400.dll and popnhax will take care of everything for me :)

v1.6 ~ back for some training and challenges ~

It's been a while~ Let's enjoy new pop'n goodness!

new options

score challenge revival

Remember "pop'n score challenge" from Peace days and earlier ? It's now back in later games, working just like before. Contact your server op to inquire about support, only if they allow patches (server-side really works the same as before, no change required).

practice mode (INSANE contribution from r2nk226 💯 )

A truly AMAZING contribution from r2nk226 which displays an interactive menu for a practice mode reminescent of console days.
This menu allows constant bpm, song slowdown, r-random

Notes:

• scores will be reset to 0 at the end of the song when this mode is active, but I still recommend only using it offline, to avoid any potential issues.
• menu doesn't display properly in SD resolution

thank you very much for your contribution 🙏

audio source fix

This was formerly known as "hdmi audio patch", I noticed an issue when trying to boot the game with a different default audio source on real hardware (especially useful if you replaced your cab hardware with a low-latency audio install)

enhanced options

quick retire: medals

Using quick retire will now grant you a "fail" medal along with "E" rank. This has been an issue with some server solutions which would erase previously obtained medals. This should avoid any future trouble.

hidden is offset: result screen enhancements

Two new options are available regarding information displayed on result screen.

• show_offset: You will see "hidden" value on result screen which might be useful for further reference
• show_fast_slow: You will see the fast and slow note counters on result screen even if judge+ was set to something else than "timing". Might be useful to help setting your offset if you're used to one of these options.

force datecode: network support

There was a bug with the previous force_datecode patch where network packets would still reflect the original date rather than the forced one.
A new network_datecode option allows to make network packets reflect this change if needed.

multiboot setups

Multiboot setups are now easier to build and maintain across popnhax updates:

if force_datecode isn't set in your .xml, and game dll is named popn22_yyyymmddrr.dll, popnhax will automatically apply a yyyymmddrr force_datecode on-the-fly.

Basically, you can keep one popnhax.xml file, and multiple popn22 dlls to have a working multiboot setup for eclale onwards.

This behavior can be disabled with disable_multiboot option.

v1.5 ~ quick retry ~

A couple new/updated options in this release, as well as a cool enhancement rework 👍

new/updated options

quick retire -> quick retry

when used in combination with pfree, quick retire brings you quick retry as well!!
Press numpad 7 during song to go back to option select screen, or hold numpad 7 to restart song
(Note: I won't bother porting this to non-pfree mode since retry bypasses the stage update counter, doesn't make much sense to keep...)

force full options

This always enables advanced options, no need to press numpad 0 (useful if you don't have a numpad on your setup)

fps uncap (debug option)

This is pulse's unlocker patch (thanks for it :) ). When running at 120fps+, a side effect from pop'n engine kicks in and caps to around 80 fps.
Enabling it allows you to remove this limit. Animations will be even more sped up than usual, but song stays in sync and the timing feels better (pop'n infamous 6 frame lag is much shorter at 120Hz, combine with borderless windowed for an even better experience)

enhancement for multiboot setups

popnhax now supports different popn dll and popnhax xml filenames, as well as a datecode override option, which means that you can run multiple versions from the same folder, you just have to keep the popn22.dll file from different versions.

--popnhax-config (commandline arg)

When launched in commandline, use -k popnhax.dll --popnhax-config filename.xml to specify a different patch file
(useful if you want to run pfree/nonpfree or omni/nonomni versions etc..)

By default, if you specify a different popn dll filename, (e.g. spice.exe -pnm popn22_peace.dll -k popnhax.dll then popn22_peace.xml takes priority over popnhax.xml
Note: the dll filename has to start with "pop2" to be recognized as the game dll

force_datecode (debug option)

This allows to specify a different datecode rather than having to edit ea3-config.xml (also works for omnimix patch file detection)

v1.4 ~ debug options ~

Always going strong with OFFSET ADJUST, FORCE HD TIMING/RESOLUTION (for SD cabs with LCD screens), PREMIUM FREE, QUICK RETIRE 💪 💪 💪

Changelog:

• xml has been rewritten (debug options are separated from the rest)
• hd_on_sd has been reworked into two separate patches: force_hd_timing and force_hd_resolution
• force_hd_resolution also reduces popkun and beam brightness (as is done on real hd mode)
• beam brightness is further adjustable as a debug setting if you wish
• keysound offset is adjustable as a debug setting (will force you to fire keysounds later/earlier)

v1.3 ~ stability update ~

Always going strong with OFFSET ADJUST, HD ON SD (for SD cabs with LCD screens), PREMIUM FREE, QUICK RETIRE 💪 💪 💪

Changelog:

• Fix pop'n engine issue on custom charts with too many notes where score would overflow
• Fix force_unlocks option causing "Failed to load popnhax.dll" especially on omnimix with latest version
• Separated deco_unlocks into its own option since later game don't have this anymore, it's safer to disable
• Fix included popnhax.xml default values (make sure to overwrite your existing .xml file and set your options back)

v1.2b ~ OFFSET ADJUST ?! ~

Don't you wish pop'n had offset adjust?? Then download this :D

ALL PATCHES HAVE BEEN TESTED WORKING FROM USANEKO ONWARDS

(Eclale also works with minor issues but there's no long notes so it's out of scope for omnimix popnhax anyways)

2 new options added:

HIDDEN IS OFFSET

This turns "hidden+" option into an offset adjust!
It can go from -70 (forcing you to hit even earlier than usual) to +220 (where popkun will sit forever on that red line before you hit)

+8/+16 is what works best for me but your mileage may vary. Having it mapped to hidden+ allows each user to customize to their taste :)

(bugfix: now working with beatpop and charapop, sorry for inconvenience)

QUICK RETIRE

• Hit numpad 9 at anytime during the song to exit directly to result screen
  (Note: obviously it will always count as a fail even if you had a clear gauge)

• Hit numpad 9 during the result screen before pressing red button and you will gracefully exit to "thank you for playing" screen
  (Note: this is especially useful when using premium free as this will allow you to exit the session and get your pop'n rank rise)

1 option enhanced:

HD ON SD (for cab owners)

Since the timing differences have been worked out, HD on SD is now a bit cleaner (no more HD lamps in test menu or on boot screen)

leave pcb_type as 0

• mode 1 will set the timing to HD
• mode 2 will set the timing to HD and force 768p resolution with centered texts

v1.2 ~ OFFSET ADJUST?! ~

Don't you wish pop'n had offset adjust?? Then download this :D

ALL PATCHES HAVE BEEN TESTED WORKING FROM USANEKO ONWARDS

(Eclale also works with minor issues but there's no long notes so it's out of scope for omnimix popnhax anyways)

1 new option added:

HIDDEN IS OFFSET

This turns "hidden+" option into an offset adjust!
It can go from -70 (forcing you to hit even earlier than usual) to +220 (where popkun will sit forever on that red line before you hit)

+8/+16 is what works best for me but your mileage may vary. Having it mapped to hidden+ allows each user to customize to their taste :)

2 options enhanced:

QUICK EXIT FOR PREMIUM FREE

This still allows an infinite number of songs per credit.

• Hit numpad 9 at anytime during the song to exit directly to result screen
  (Note: obviously it will always count as a fail even if you had a clear gauge)

• Hit numpad 9 during the result screen before pressing red button and you will gracefully exit to "thank you for playing" screen

HD ON SD (for cab owners)

Since the timing differences have been worked out, HD on SD is now a bit cleaner (no more HD lamps in test menu or on boot screen)

leave pcb_type as 0

• mode 1 will set the timing to HD
• mode 2 will set the timing to HD and force 768p resolution with centered texts

v1.1b ~ pfree edition ~

New patches for Pop'n :D Also a good time to finally put this project on github!

ALL PATCHES HAVE BEEN TESTED WORKING FROM USANEKO ONWARDS

2 new options supported:

PREMIUM FREE

This allows an infinite number of songs per credit.
Note that this will prevent you from getting your pop'n class raise, but I left a workaround if you want to get your gainz

workaround

Setting the game to 1 song per credit will override the pfree patch, which allows this workflow:

• Play premium free as long as you want
• Test menu -> game options -> 1 song per credit -> save and exit -> game mode
• Play a one song credit

(Note: even if you don't use the workaround, eventually when you disable pfree you'll get all your rise, with a cap of 2.00 points per credit, but it'll keep growing even if you fail lvl 1 songs...)

HD ON SD (for cab owners)

HD and SD pop'n seem to have different internal offset to account for LCD additional latency.

You can set this option to 1 and the game will always run at 640x480 resolution even if you set your pcb_type to 1 or 2 (and you'll get J:A:A / J:C:A model name and offset).

You can also set this option to 2 and the game will run at 1360x768 but with model name, credit number and network status texts centered (this is perfect for a SD cab with a 32" LCD modded in to keep the same active 4:3 surface)

(Note: original SD BemaniPC seems to struggle a bit with running the game in HD.. I recommend changing the internals anyways :D )