Update on Readme for depth perspective compensation

davidpagnon · davidpagnon · commit 27507e32072b · 2025-11-13T01:27:48.000+01:00
diff --git a/README.md b/README.md
@@ -24,7 +24,7 @@
 </br>
 
 > **`Announcements:`**
-> - Generate or import a calibration file, OpenSim skeleton overlay **New in v0.9!** 
+> - Compensate for floor angle, floor height, depth perspective effects, generate a calibration file **New in v0.9!** 
 > - Select only the persons you want to analyze **New in v0.8!** 
 > - MarkerAugmentation and Inverse Kinematics for accurate 3D motion with OpenSim. **New in v0.7!** 
 > - Any detector and pose estimation model can be used. **New in v0.6!**
@@ -266,30 +266,35 @@ sports2d --person_ordering_method on_click
 #### Get coordinates in meters: 
 > **N.B.:** The Z coordinate (depth) should not be overly trusted. 
 
-You may want coordinates in meters rather than pixels. 2 options to do so:
+To convert from pixels to meters, you need a minima the height of a participant. Better results can be obtained by also providing an information on depth. The camera horizon angle and the floor height are generally automatically estimated. **N.B.: A calibration file will be generated.** 
 
-1. **Just provide the height of a reference person**:
-   - Their height in meters is be compared with their height in pixels to get a pixel-to-meter conversion factor.
-   - To estimate the depth coordinates, specify which side of the person is visible: `left`, `right`, `front`, or `back`. Use `auto` if you want it to be automatically determined (only works for motions in the sagittal plane), or `none` if you want to keep 2D coordinates instead of 3D (if the person turns around, for example).
-   - The floor angle is automatically estimated from gait, as well as the origin of the xy axis. The person trajectory is corrected accordingly. You can use the `--floor_angle` and `--xy_origin` parameters to manually specify them if your subject is not travelling horizontally or if you want the origin not to be under their feet (note that the `y` axis points down).
-   
-   **N.B.: A calibration file will be generated.** By convention, the camera-to-subject distance is set to 10 meters.
+- The pixel-to-meters scale is computed from the ratio between the height of the participant in meters and in pixels. The height in pixels is automatically calculated; use the `--first_person_height` parameter to specify the height in meters.
+- Depth perspective effects can be compensated either with the camera-to-person distance (m), or focal length (px), or field-of-view (degrees or radians), or from a calibration file. Use the `--perspective_unit` ('distance_m', 'f_px', 'fov_deg', 'fov_rad', or 'from_calib') and `--perspective_value` parameters (resp. in m, px, deg, rad, or '').
+- The camera horizon angle can be estimated from kinematics (`auto`), from a calibration file (`from_calib`), or manually (float). Use the `--floor_angle` parameter.
+- Likewise for the floor level. Use the `--xy_origin` parameter.
 
-   ``` cmd
-   sports2d --first_person_height 1.65 --visible_side auto front none
-   ```
-   ``` cmd
-   sports2d --first_person_height 1.65 --visible_side auto front none `
-            --person_ordering_method on_click `
-            --floor_angle 0 --xy_origin 0 940
-   ```
+If one of these parameters is set to `from_calib`, then use `--calib_file`.
 
-2. **Or use a calibration file**:\
-  It can either be a `.toml` calibration file previously generated by Sports2D, or a more accurate one coming from another system. For example, [Pose2Sim](https://github.com/perfanalytics/pose2sim) can be used to accurately calculate calibration, or to convert calibration files from Qualisys, Vicon, OpenCap, FreeMoCap, etc.
 
-   ``` cmd
-   sports2d --calib_file Calib_demo.toml --visible_side auto front none
-   ```
+``` cmd
+sports2d --first_person_height 1.65
+```
+``` cmd
+sports2d --first_person_height 1.65 `
+        --floor_angle auto `
+        --xy_origin auto`
+        --perspective_unit distance_m --perspective_value 10
+```
+``` cmd
+sports2d --first_person_height 1.65 `
+        --floor_angle 0 `
+        --xy_origin from_calib`
+        --perspective_unit from_calib --calib_file Sports2D\Demo\Calib_demo.toml
+```
+``` cmd
+sports2d --first_person_height 1.65 `
+        --perspective_unit f_px --perspective_value 2520
+```
 
 <br>
 
@@ -452,20 +457,25 @@ sports2d --help
 'config': ["C", "path to a toml configuration file"],
 
 'video_input': ["i", "webcam, or video_path.mp4, or video1_path.avi video2_path.mp4 ... Beware that images won't be saved if paths contain non ASCII characters"],
+'time_range': ["t", "start_time end_time. In seconds. Whole video if not specified. start_time1 end_time1 start_time2 end_time2 ... if multiple videos with different time ranges"],
 'nb_persons_to_detect': ["n", "number of persons to detect. int or 'all'. 'all' if not specified"],
 'person_ordering_method': ["", "'on_click', 'highest_likelihood', 'largest_size', 'smallest_size', 'greatest_displacement', 'least_displacement', 'first_detected', or 'last_detected'. 'on_click' if not specified"],
 'first_person_height': ["H", "height of the reference person in meters. 1.65 if not specified. Not used if a calibration file is provided"],
 'visible_side': ["", "front, back, left, right, auto, or none. 'auto front none' if not specified. If 'auto', will be either left or right depending on the direction of the motion. If 'none', no IK for this person"],
+'participant_mass': ["", "mass of the participant in kg or none. Defaults to 70 if not provided. No influence on kinematics (motion), only on kinetics (forces)"],
+'perspective_value': ["", "Either camera-to-person distance (m), or focal length (px), or field-of-view (degrees or radians), or '' if perspective_unit=='from_calib'"],
+'perspective_unit': ["", "'distance_m', 'f_px', 'fov_deg', 'fov_rad', or 'from_calib'"],
+'do_ik': ["", "do inverse kinematics. false if not specified"],
+'use_augmentation': ["", "Use LSTM marker augmentation. false if not specified"],
 'load_trc_px': ["", "load trc file to avaid running pose estimation again. false if not specified"],
 'compare': ["", "visually compare motion with trc file. false if not specified"],
-'webcam_id': ["w", "webcam ID. 0 if not specified"],
-'time_range': ["t", "start_time end_time. In seconds. Whole video if not specified. start_time1 end_time1 start_time2 end_time2 ... if multiple videos with different time ranges"],
 'video_dir': ["d", "current directory if not specified"],
 'result_dir': ["r", "current directory if not specified"],
+'webcam_id': ["w", "webcam ID. 0 if not specified"],
 'show_realtime_results': ["R", "show results in real-time. true if not specified"],
 'display_angle_values_on': ["a", '"body", "list", "body" "list", or "none". body list if not specified'],
 'show_graphs': ["G", "show plots of raw and processed results. true if not specified"],
-'save_graphs': ["", "save position and angle plots of raw and processed results. false if not specified"],
+'save_graphs': ["", "save position and angle plots of raw and processed results. true if not specified"],
 'joint_angles': ["j", '"Right ankle" "Left ankle" "Right knee" "Left knee" "Right hip" "Left hip" "Right shoulder" "Left shoulder" "Right elbow" "Left elbow" if not specified'],
 'segment_angles': ["s", '"Right foot" "Left foot" "Right shank" "Left shank" "Right thigh" "Left thigh" "Pelvis" "Trunk" "Shoulders" "Head" "Right arm" "Left arm" "Right forearm" "Left forearm" if not specified'],
 'save_vid': ["V", "save processed video. true if not specified"],
@@ -486,11 +496,8 @@ sports2d --help
 'xy_origin': ["", "origin of the xy plane. 'auto' if not specified"],
 'calib_file': ["", "path to calibration file. '' if not specified, eg no calibration file"],
 'save_calib': ["", "save calibration file. true if not specified"],
-'do_ik': ["", "do inverse kinematics. false if not specified"],
-'use_augmentation': ["", "Use LSTM marker augmentation. false if not specified"],
 'feet_on_floor': ["", "offset marker augmentation results so that feet are at floor level. true if not specified"],
-'use_simple_model': ["", "IK 10+ times faster, but no muscles or flexible spine. false if not specified"],
-'participant_mass': ["", "mass of the participant in kg or none. Defaults to 70 if not provided. No influence on kinematics (motion), only on kinetics (forces)"],
+'use_simple_model': ["", "IK 10+ times faster, but no muscles or flexible spine, no patella. false if not specified"],
 'close_to_zero_speed_m': ["","Sum for all keypoints: about 50 px/frame or 0.2 m/frame"], 
 'tracking_mode': ["", "'sports2d' or 'deepsort'. 'deepsort' is slower, harder to parametrize but can be more robust if correctly tuned"],
 'deepsort_params': ["", 'Deepsort tracking parameters: """{dictionary between 3 double quotes}""". \n\
@@ -500,6 +507,7 @@ sports2d --help
 'keypoint_likelihood_threshold': ["", "detected keypoints are not retained if likelihood is below this threshold. 0.3 if not specified"],
 'average_likelihood_threshold': ["", "detected persons are not retained if average keypoint likelihood is below this threshold. 0.5 if not specified"],
 'keypoint_number_threshold': ["", "detected persons are not retained if number of detected keypoints is below this threshold. 0.3 if not specified, i.e., i.e., 30 percent"],
+'max_distance': ["", "If a person is detected further than max_distance from its position on the previous frame, it will be considered as a new one. in px or None, 100 by default."],
 'fastest_frames_to_remove_percent': ["", "Frames with high speed are considered as outliers. Defaults to 0.1"],
 'close_to_zero_speed_px': ["", "Sum for all keypoints: about 50 px/frame or 0.2 m/frame. Defaults to 50"],
 'large_hip_knee_angles': ["", "Hip and knee angles below this value are considered as imprecise. Defaults to 45"],
@@ -511,15 +519,16 @@ sports2d --help
 'interp_gap_smaller_than': ["", "interpolate sequences of missing data if they are less than N frames long. 10 if not specified"],
 'fill_large_gaps_with': ["", "last_value, nan, or zeros. last_value if not specified"],
 'sections_to_keep': ["", "all, largest, first, or last.  Keep 'all' valid sections even when they are interspersed with undetected chunks, or the 'largest' valid section, or the 'first' one, or the 'last' one"],
+'min_chunk_size': ["", "Minimum number of valid frames in a row to keep a chunk of data for a person.  10 if not specified"],
 'reject_outliers': ["", "reject outliers with Hampel filter before other filtering methods. true if not specified"],
 'filter': ["", "filter results. true if not specified"],
 'filter_type': ["", "butterworth, kalman, gcv_spline, gaussian, median, or loess. butterworth if not specified"],
+'cut_off_frequency': ["", "cut-off frequency of the Butterworth filter. 6 if not specified"],
 'order': ["", "order of the Butterworth filter. 4 if not specified"],
-'cut_off_frequency': ["", "cut-off frequency of the Butterworth filter. 3 if not specified"],
+'gcv_cut_off_frequency': ["", "cut-off frequency of the GCV spline filter. 'auto' is usually better, unless the signal is too short (noise can then be considered as signal -> trajectories not filtered). 'auto' if not specified"],
+'gcv_smoothing_factor': ["", "smoothing factor of the GCV spline filter (>=0). Ignored if cut_off_frequency != 'auto'. Biases results towards more smoothing (>1) or more fidelity to data (<1). 1.0 if not specified"],
 'trust_ratio': ["", "trust ratio of the Kalman filter: How much more do you trust triangulation results (measurements), than the assumption of constant acceleration(process)? 500 if not specified"],
 'smooth': ["", "dual Kalman smoothing. true if not specified"],
-'gcv_cut_off_frequency': ["", "cut-off frequency of the GCV spline filter. 'auto' if not specified"],
-'smoothing_factor': ["", "smoothing factor of the GCV spline filter (>=0). Ignored if cut_off_frequency != 'auto'. Biases results towards more smoothing (>1) or more fidelity to data (<1). 0.1 if not specified"],
 'sigma_kernel': ["", "sigma of the gaussian filter. 1 if not specified"],
 'nb_values_used': ["", "number of values used for the loess filter. 5 if not specified"],
 'kernel_size': ["", "kernel size of the median filter. 3 if not specified"],
@@ -628,11 +637,11 @@ Sports2D:
 
 2. **Sets up pose estimation with RTMLib.** It can be run in lightweight, balanced, or performance mode, and for faster inference, the person bounding boxes can be tracked instead of detected every frame. Any RTMPose model can be used. 
 
-3. **Tracks people** so that their IDs are consistent across frames. A person is associated to another in the next frame when they are at a small distance. IDs remain consistent even if the person disappears from a few frames. We crafted a 'sports2D' tracker which gives good results and runs in real time, but it is also possible to use `deepsort` in particularly challenging situations. 
+3. **Tracks people** so that their IDs are consistent across frames. A person is associated to another in the next frame when they are at a small distance. IDs remain consistent even if the person disappears from a few frames, thanks to the 'sports2D' tracker. [See Release notes of v0.8.22 for more information](https://github.com/davidpagnon/Sports2D/releases/tag/v0.8.22). 
 
 4. **Chooses which persons to analyze.** In single-person mode, only keeps the person with the highest average scores over the sequence. In multi-person mode, you can choose the number of persons to analyze (`nb_persons_to_detect`), and how to order them (`person_ordering_method`). The ordering method can be 'on_click', 'highest_likelihood', 'largest_size', 'smallest_size', 'greatest_displacement', 'least_displacement', 'first_detected', or 'last_detected'. `on_click` is default and lets the user click on the persons they are interested in, in the desired order.
 
-4. **Converts the pixel coordinates to meters.** The user can provide the size of a specified person to scale results accordingly. The floor angle and the coordinate origin can either be detected automatically from the gait sequence, or be manually specified. The depth coordinates are set to normative values, depending on whether the person is going left, right, facing the camera, or looking away.
+4. **Converts the pixel coordinates to meters.** The user can provide the size of a specified person to scale results accordingly. The camera horizon angle and the floor level can either be detected automatically from the gait sequence, be manually specified, or obtained frmm a calibration file. The depth perspective effects are compensated thanks with the distance from the camera to the subject, the focal length, the field of view, or from a calibration file. [See Release notes of v0.8.25 for more information](https://github.com/davidpagnon/Sports2D/releases/tag/v0.8.25). 
 
 5. **Computes the selected joint and segment angles**, and flips them on the left/right side if the respective foot is pointing to the left/right. 
 
