Strange behavior in cartpole.py #5
Description
Hi,
When running cartpole.py, I get the following (go to step 119):
% python cartpole.py
INFO:evograd.noise:Sampling 10000000 random numbers with seed 42
INFO:evograd.noise:Sampled 40000000 bytes
/Users/tcf/anaconda3/envs/evo/lib/python3.7/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
step: 0, mean fitness: 120.77
step: 1, mean fitness: 124.54
step: 2, mean fitness: 132.03
step: 3, mean fitness: 136.29
step: 4, mean fitness: 137.41
step: 5, mean fitness: 126.86
step: 6, mean fitness: 146.96
step: 7, mean fitness: 150.17
step: 8, mean fitness: 154.71
step: 9, mean fitness: 152.27
step: 10, mean fitness: 159.49
step: 11, mean fitness: 163.4
step: 12, mean fitness: 162.69
step: 13, mean fitness: 159.5
step: 14, mean fitness: 150.78
step: 15, mean fitness: 165.73
step: 16, mean fitness: 169.43
step: 17, mean fitness: 176.08
step: 18, mean fitness: 166.3
step: 19, mean fitness: 178.74
step: 20, mean fitness: 181.06
step: 21, mean fitness: 181.42
step: 22, mean fitness: 182.07
step: 23, mean fitness: 175.19
step: 24, mean fitness: 181.09
step: 25, mean fitness: 178.61
step: 26, mean fitness: 184.12
step: 27, mean fitness: 185.27
step: 28, mean fitness: 184.36
step: 29, mean fitness: 181.19
step: 30, mean fitness: 184.31
step: 31, mean fitness: 182.96
step: 32, mean fitness: 188.73
step: 33, mean fitness: 188.25
step: 34, mean fitness: 189.97
step: 35, mean fitness: 193.47
step: 36, mean fitness: 191.24
step: 37, mean fitness: 192.58
step: 38, mean fitness: 187.92
step: 39, mean fitness: 188.66
step: 40, mean fitness: 189.48
step: 41, mean fitness: 191.93
step: 42, mean fitness: 191.03
step: 43, mean fitness: 191.72
step: 44, mean fitness: 191.17
step: 45, mean fitness: 192.76
step: 46, mean fitness: 192.89
step: 47, mean fitness: 191.14
step: 48, mean fitness: 196.5
step: 49, mean fitness: 194.53
step: 50, mean fitness: 191.08
step: 51, mean fitness: 193.84
step: 52, mean fitness: 195.39
step: 53, mean fitness: 194.18
step: 54, mean fitness: 191.56
step: 55, mean fitness: 193.55
step: 56, mean fitness: 195.62
step: 57, mean fitness: 190.07
step: 58, mean fitness: 194.5
step: 59, mean fitness: 199.6
step: 60, mean fitness: 195.21
step: 61, mean fitness: 193.79
step: 62, mean fitness: 194.38
step: 63, mean fitness: 198.5
step: 64, mean fitness: 198.08
step: 65, mean fitness: 197.03
step: 66, mean fitness: 193.46
step: 67, mean fitness: 195.71
step: 68, mean fitness: 197.98
step: 69, mean fitness: 192.48
step: 70, mean fitness: 198.83
step: 71, mean fitness: 199.3
step: 72, mean fitness: 197.35
step: 73, mean fitness: 199.26
step: 74, mean fitness: 198.08
step: 75, mean fitness: 195.37
step: 76, mean fitness: 199.07
step: 77, mean fitness: 198.37
step: 78, mean fitness: 199.12
step: 79, mean fitness: 194.08
step: 80, mean fitness: 198.94
step: 81, mean fitness: 197.03
step: 82, mean fitness: 199.53
step: 83, mean fitness: 198.38
step: 84, mean fitness: 195.94
step: 85, mean fitness: 198.77
step: 86, mean fitness: 199.41
step: 87, mean fitness: 195.99
step: 88, mean fitness: 197.38
step: 89, mean fitness: 195.78
step: 90, mean fitness: 197.69
step: 91, mean fitness: 199.55
step: 92, mean fitness: 199.54
step: 93, mean fitness: 197.72
step: 94, mean fitness: 197.13
step: 95, mean fitness: 192.07
step: 96, mean fitness: 199.21
step: 97, mean fitness: 197.8
step: 98, mean fitness: 199.78
step: 99, mean fitness: 198.93
step: 100, mean fitness: 199.01
step: 101, mean fitness: 199.42
step: 102, mean fitness: 199.06
step: 103, mean fitness: 199.55
step: 104, mean fitness: 193.92
step: 105, mean fitness: 198.15
step: 106, mean fitness: 199.96
step: 107, mean fitness: 199.94
step: 108, mean fitness: 198.41
step: 109, mean fitness: 196.36
step: 110, mean fitness: 199.65
step: 111, mean fitness: 199.8
step: 112, mean fitness: 199.69
step: 113, mean fitness: 199.12
step: 114, mean fitness: 196.0
step: 115, mean fitness: 199.94
step: 116, mean fitness: 199.6
step: 117, mean fitness: 199.88
step: 118, mean fitness: 200.0
step: 119, mean fitness: 9.352
step: 120, mean fitness: 9.36
step: 121, mean fitness: 9.332
step: 122, mean fitness: 9.332
step: 123, mean fitness: 9.302
step: 124, mean fitness: 9.346
step: 125, mean fitness: 9.346
step: 126, mean fitness: 9.372
step: 127, mean fitness: 9.326
step: 128, mean fitness: 9.302
step: 129, mean fitness: 9.378
step: 130, mean fitness: 9.376
step: 131, mean fitness: 9.338
step: 132, mean fitness: 9.396
step: 133, mean fitness: 9.336
step: 134, mean fitness: 9.388
step: 135, mean fitness: 9.344
step: 136, mean fitness: 9.356
step: 137, mean fitness: 9.326
step: 138, mean fitness: 9.338
step: 139, mean fitness: 9.376
step: 140, mean fitness: 9.354
step: 141, mean fitness: 9.314
step: 142, mean fitness: 9.35
step: 143, mean fitness: 9.4
step: 144, mean fitness: 9.386
step: 145, mean fitness: 9.38
step: 146, mean fitness: 9.312
step: 147, mean fitness: 9.334
step: 148, mean fitness: 9.374
step: 149, mean fitness: 9.302
step: 150, mean fitness: 9.324
step: 151, mean fitness: 9.39
step: 152, mean fitness: 9.342
step: 153, mean fitness: 9.372
step: 154, mean fitness: 9.38
step: 155, mean fitness: 9.428
step: 156, mean fitness: 9.41
step: 157, mean fitness: 9.346
step: 158, mean fitness: 9.366
step: 159, mean fitness: 9.384
step: 160, mean fitness: 9.348
step: 161, mean fitness: 9.282
step: 162, mean fitness: 9.312
step: 163, mean fitness: 9.338
step: 164, mean fitness: 9.32
step: 165, mean fitness: 9.39
step: 166, mean fitness: 9.302
step: 167, mean fitness: 9.374
step: 168, mean fitness: 9.356
step: 169, mean fitness: 9.33
step: 170, mean fitness: 9.356
step: 171, mean fitness: 9.366
step: 172, mean fitness: 9.41
step: 173, mean fitness: 9.366
step: 174, mean fitness: 9.364
step: 175, mean fitness: 9.378
step: 176, mean fitness: 9.4
step: 177, mean fitness: 9.386
step: 178, mean fitness: 9.43
step: 179, mean fitness: 9.374
step: 180, mean fitness: 9.42
step: 181, mean fitness: 9.316
step: 182, mean fitness: 9.31
step: 183, mean fitness: 9.348
step: 184, mean fitness: 9.384
step: 185, mean fitness: 9.396
step: 186, mean fitness: 9.436
step: 187, mean fitness: 9.36
step: 188, mean fitness: 9.32
step: 189, mean fitness: 9.396
step: 190, mean fitness: 9.34
step: 191, mean fitness: 9.328
step: 192, mean fitness: 9.354
step: 193, mean fitness: 9.408
step: 194, mean fitness: 9.334
step: 195, mean fitness: 9.346
step: 196, mean fitness: 9.388
step: 197, mean fitness: 9.366
step: 198, mean fitness: 9.318
step: 199, mean fitness: 9.358
step: 200, mean fitness: 9.412
step: 201, mean fitness: 9.308
step: 202, mean fitness: 9.338
step: 203, mean fitness: 9.406
step: 204, mean fitness: 9.392
step: 205, mean fitness: 9.388
step: 206, mean fitness: 9.44
step: 207, mean fitness: 9.344
step: 208, mean fitness: 9.384
step: 209, mean fitness: 9.338
step: 210, mean fitness: 9.272
step: 211, mean fitness: 9.36
step: 212, mean fitness: 9.326
step: 213, mean fitness: 9.404
step: 214, mean fitness: 9.416
step: 215, mean fitness: 9.31
step: 216, mean fitness: 9.364
step: 217, mean fitness: 9.334
step: 218, mean fitness: 9.376
step: 219, mean fitness: 9.364
step: 220, mean fitness: 9.296
step: 221, mean fitness: 9.344
step: 222, mean fitness: 9.354
step: 223, mean fitness: 9.384
step: 224, mean fitness: 9.284
step: 225, mean fitness: 9.382
step: 226, mean fitness: 9.344
step: 227, mean fitness: 9.354
step: 228, mean fitness: 9.364
step: 229, mean fitness: 9.346
step: 230, mean fitness: 9.308
step: 231, mean fitness: 9.332
step: 232, mean fitness: 9.326
step: 233, mean fitness: 9.344
step: 234, mean fitness: 9.332
step: 235, mean fitness: 9.334
step: 236, mean fitness: 9.356
step: 237, mean fitness: 9.328
step: 238, mean fitness: 9.346
step: 239, mean fitness: 9.286
step: 240, mean fitness: 9.354
step: 241, mean fitness: 9.374
step: 242, mean fitness: 9.372
step: 243, mean fitness: 9.376
step: 244, mean fitness: 9.328
step: 245, mean fitness: 9.354
step: 246, mean fitness: 9.382
step: 247, mean fitness: 9.328
step: 248, mean fitness: 9.346
step: 249, mean fitness: 9.362
step: 250, mean fitness: 9.362
step: 251, mean fitness: 9.324
step: 252, mean fitness: 9.354
step: 253, mean fitness: 9.36
step: 254, mean fitness: 9.366
step: 255, mean fitness: 9.4
step: 256, mean fitness: 9.392
step: 257, mean fitness: 9.352
step: 258, mean fitness: 9.348
step: 259, mean fitness: 9.372
step: 260, mean fitness: 9.312
step: 261, mean fitness: 9.33
step: 262, mean fitness: 9.396
step: 263, mean fitness: 9.36
step: 264, mean fitness: 9.34
step: 265, mean fitness: 9.338
step: 266, mean fitness: 9.354
step: 267, mean fitness: 9.246
step: 268, mean fitness: 9.404
step: 269, mean fitness: 9.342
step: 270, mean fitness: 9.302
step: 271, mean fitness: 9.302
step: 272, mean fitness: 9.354
step: 273, mean fitness: 9.37
step: 274, mean fitness: 9.316
step: 275, mean fitness: 9.318
step: 276, mean fitness: 9.34
step: 277, mean fitness: 9.344
step: 278, mean fitness: 9.348
step: 279, mean fitness: 9.338
step: 280, mean fitness: 9.382
step: 281, mean fitness: 9.322
step: 282, mean fitness: 9.324
step: 283, mean fitness: 9.396
step: 284, mean fitness: 9.394
step: 285, mean fitness: 9.384
step: 286, mean fitness: 9.428
step: 287, mean fitness: 9.412
step: 288, mean fitness: 9.398
step: 289, mean fitness: 9.332
step: 290, mean fitness: 9.326
step: 291, mean fitness: 9.354
step: 292, mean fitness: 9.374
step: 293, mean fitness: 9.368
step: 294, mean fitness: 9.312
step: 295, mean fitness: 9.39
step: 296, mean fitness: 9.41
step: 297, mean fitness: 9.372
step: 298, mean fitness: 9.342
step: 299, mean fitness: 9.35
step: 300, mean fitness: 9.386
step: 301, mean fitness: 9.338
Is this supposed to happen while maximizing mean fitness? If so, why?