Conversation
…ecute-phase-1 Initialize repository structure
…ne-next-steps Initial implementation with license and tests
…-port-plan Implement small utilities and update plan
…anned Add duotrio_pc function
…t-as-planned Add discrim_2afc function
…t-as-planned Add duotrio_pc function
This commit includes the following major additions and enhancements to the sensPy library, progressing significantly on the plan to port sensR:
1. **BetaBinomial Model (`senspy.models`):**
* Full implementation of the `BetaBinomial` class with a `fit` method using Numba-optimized log-likelihood functions (standard and chance-corrected).
* Added `summary()` for model output and `confint()` for profile likelihood confidence intervals.
* Validated with Python-specific unit tests; performance benchmarks meet targets.
2. **Discrimination Functions (`senspy.discrimination`):**
* Implemented a general `discrim` function for d-prime estimation from various methods (2AFC, Triangle, Duotrio, etc.), including Wald SE, CI, and p-values.
* Implemented the `twoAC` function for estimating delta (sensitivity) and tau (bias) in 2-Alternative Choice tasks.
* Added comprehensive unit tests for these functions (Python-specific tests passing).
3. **Link Utilities (`senspy.links`):**
* Generalized `psyfun`, `psyinv`, and `psyderiv` to be method-aware, supporting various discrimination protocols.
* Significantly enhanced `rescale` to use these generalized functions, support different methods, and perform standard error transformations via the delta method.
4. **Utilities and Testing (`senspy.utils`, `tests/`):**
* Reviewed existing utilities.
* Expanded test suites for models and discrimination functions. R-comparison tests are structurally in place but skipped due to sandbox environment limitations with rpy2.
This work completes a major phase of porting sensR's core statistical capabilities for sensory discrimination analysis to Python.
feat: Implement core sensR functionalities in sensPy
…tation site This commit delivers the implementation of several core sensory discrimination methodologies and statistical tests previously missing from sensPy, based on a final review of the sensR package. It also initializes a new Eleventy-based documentation site. Key functional additions to sensPy: - Degree of Difference (DoD) method: Full ML estimation. - Same-Different (SD) method: Full ML estimation. - dprime_test: Hypothesis testing for a common d-prime against a value. - dprime_compare: Likelihood Ratio Test for equality of multiple d-primes. - SDT: d-prime estimation from rating scale data (probit/logit). - AUC: Calculation of Area Under ROC Curve from d-prime. - Basic unit tests have been added for all new functionalities. Documentation (`documentation/` directory): - Initialized a new Eleventy project. - Configured basic directory structure (src, _includes, _site). - Added Eleventy configuration (`eleventy.config.js`). - Created a minimal `index.md` and a base Nunjucks layout. - Set up npm scripts for building and serving the documentation site. - Added a .gitignore file for the documentation directory. This completes the planned functional porting for the current phase and sets up the foundation for comprehensive documentation.
feat: Implement core sensR functionalities in sensPy
This commit introduces a new Eleventy-based documentation site in the `documentation/` directory. Key features: - Initialized Eleventy project with configuration for input/output, templates (Nunjucks), and static asset passthrough (CSS). - Developed a core site structure including a base layout (`base.njk`) with a sidebar for navigation and a main content area. - Implemented basic CSS for clean and readable presentation of text, code blocks, and navigation elements. - Created a JSON data file (`sensr_senspy_mapping.json`) to map original `sensR` functions to their `senspy` equivalents, ensuring consistency. Detailed documentation pages have been created for all major `senspy` functionalities, including: - `senspy.models.BetaBinomial`: Introduction, fitting, attributes, methods (`summary`, `confint`), and usage examples. - `senspy.discrimination.discrim`: General d-prime estimation, parameters, return structure, examples for various methods. - `senspy.discrimination.twoAC`: 2-AFC model, parameters, model details, return structure, examples. - `senspy.discrimination.dod`: Degree of Difference model, parameters, model details, return structure, examples. - `senspy.discrimination.samediff`: Same-Different model, parameters, model details, return structure, examples. - `senspy.discrimination.dprime_test` & `dprime_compare`: Hypothesis testing for d-prime values, parameters, return structures, examples. - `senspy.discrimination.SDT` & `AUC`: ROC utilities for d-prime from rating scales and Area Under Curve calculation, parameters, examples. - `senspy.links` utilities (`psyfun`, `psyinv`, `psyderiv`, `rescale`): Core psychometric conversions, method-aware implementations, and standard error rescaling, with examples. All documentation pages include explanations of purpose, parameters, return values, `sensR` equivalents, and runnable Python code examples. The site structure, navigation, and content have been reviewed for clarity and accuracy.
docs: Create and populate Eleventy documentation site for sensPy
This commit delivers a significantly enhanced documentation site for sensPy,
built with Eleventy. It includes new tutorial content, elaborated examples
for all core API functions, a mapping guide from sensR, and refined site
navigation.
Key additions and improvements to the `documentation/` directory:
1. **"Getting Started" Tutorial (`tutorials/getting_started.md`):**
* Added a new tutorial providing a narrative walkthrough of a common
sensory analysis workflow using `senspy.discrimination.discrim` and
`senspy.links.rescale`.
2. **Elaborated API Examples:**
* Expanded the "Usage Examples" sections for all core `senspy` functions:
* `models.BetaBinomial`
* `discrimination.discrim`
* `discrimination.twoAC`
* `discrimination.dod`
* `discrimination.samediff`
* `discrimination.dprime_test` & `dprime_compare`
* `discrimination.SDT` & `AUC` (ROC Utilities)
* `links.psyfun`, `psyinv`, `psyderiv`, `rescale`
* New examples cover more varied scenarios, parameter combinations,
edge cases, and offer deeper interpretation of results.
3. **Illustrative Plot Placeholders:**
* Added TODO comments and descriptive text as placeholders for
illustrative plots (ROC curve, psychometric function) in
`roc_utils.md` and `links/utils.md`. Image generation scripts
are prepared but I'm currently encountering an issue with their execution.
4. **"Mapping from `sensR`" Page (`guides/mapping_from_sensr.md`):**
* Created a new guide that dynamically generates a table from
`_data/sensr_senspy_mapping.json`, mapping original `sensR`
functions to their `senspy` equivalents.
5. **Refined Navigation and Homepage:**
* Updated the sidebar navigation in `_includes/layouts/base.njk` to
include new "Tutorials" and "Guides" sections, and a more
structured "API Reference."
* Enhanced the `index.md` homepage with clear entry points to these
new sections and the API docs.
The documentation site has been reviewed for clarity, accuracy,
link integrity, and overall presentation.
…ration This commit delivers the final set of critically missed core sensR functions, completes a major enhancement pass on the Eleventy documentation site, and includes scripts for generating illustrative plots (though execution was blocked by environment issues). Core Functionality Added to sensPy: - Degree of Difference (DoD) method (`senspy.discrimination.dod`) - Same-Different (SD) method (`senspy.discrimination.samediff`) - d-prime hypothesis tests (`dprime_test`, `dprime_compare` in `senspy.discrimination`) - SDT function for d-prime from rating scales (`senspy.discrimination.SDT`) - AUC function for Area Under ROC Curve (`senspy.discrimination.AUC`) - Basic unit tests for all these new functions. Documentation Enhancements (`documentation/` directory): - Created a "Getting Started" tutorial. - Significantly elaborated usage examples for all existing API documentation pages (BetaBinomial, discrim, twoAC, DoD, Same-Different, d-prime tests, ROC utils, link utils). - Created a "Mapping from sensR" page that dynamically generates a table from a JSON data file. - Refined sidebar navigation and the homepage for better usability. - Prepared Python scripts (`documentation/scripts/`) to generate illustrative ROC and psychometric function plots using matplotlib. - Configured Eleventy to handle images. - Added placeholders in the Markdown for these plots, as I faced persistent path resolution issues during script execution. The sensPy library is now functionally more complete, and the documentation is comprehensive, providing a strong foundation for you.
This commit finalizes the written content for the sensPy documentation site, including a "Getting Started" tutorial, a "Mapping from sensR" guide, and elaborated examples for all core API functions. It also includes Python scripts (`documentation/scripts/`) intended to generate illustrative plots (ROC curve, psychometric function) for the documentation using matplotlib. The Eleventy configuration has been updated to include these images. However, due to persistent path resolution issues, I was unable to generate the plot images themselves during this session. The Markdown files have been prepared to include these images (or contain placeholders for them) once the generation step can be successfully executed. Key changes: - Completed all planned documentation text for models, discrimination functions, d-prime tests, ROC utilities, and link utilities. - Added a "Getting Started" tutorial and a "Mapping from sensR" guide. - Refined site navigation and homepage. - Created Python scripts (`generate_roc_plot.py`, `generate_psycho_plot.py`) in `documentation/scripts/` to create visual aids. - Updated Eleventy config for image passthrough and updated Markdown to reference the intended image paths. - The documentation site is structurally complete and rich in content, pending the successful generation and inclusion of the plot images.
catch up with master
This commit introduces the new `senspy.power` module, providing foundational tools for statistical power and sample size calculations related to binomial tests, which are crucial for sensory discrimination testing. Key functions implemented: - `find_critical_binomial_value`: Determines the critical number of successes for a binomial test, analogous to `sensR::findcr`. - `exact_binomial_power`: Calculates the exact statistical power for a one-sample binomial test using the critical value. - `sample_size_for_binomial_power`: Estimates the sample size required to achieve a target power for a binomial test, using an iterative search. - `power_discrim`: A general function to calculate power for various sensory discrimination methods (e.g., "2afc", "triangle") by converting d-prime values to proportions correct and then utilizing `exact_binomial_power`. Unit tests for these new functions have been written and added to `tests/test_power.py`. However, due to persistent issues with the script execution environment (`pytest` failing to find the test file), these tests could not be run to confirm their pass/fail status during this session. The implemented power functions are based on established statistical principles and `sensR`'s logic. This commit represents the initial phase of building out `sensPy`'s power analysis capabilities.
## Critical Fix
- senspy/dprime_tests.py: Corrected inverse variance weighted SE formula
- Was: se_exp = sqrt(sum(w_prime^2 * w)) - mathematically incorrect
- Now: se_exp = sqrt(1 / sum(1/variance)) - correct IVW formula
- The weighted mean formula was already correct, only SE was wrong
## Medium Priority Fix
- senspy/dprime_tests.py: Expanded valid_protocols list
- Added: "hexad", "twofive", "twofiveF"
- Now supports all protocols implemented in senspy.links module
## Test Results
355 passed, 9 xfailed, 32 warnings
## High Priority Fix
- senspy/dprime_tests.py: Replaced broken Compact Letter Display algorithm
- Old: Appended letters (e.g., 'a' → 'ab'), but shared 'a' implied no difference
- New: Greedy graph coloring ensures significantly different groups get
different letters, non-significant groups may share letters
- Groups that ARE significantly different now correctly get different letters
## Medium Priority Fix
- senspy/anota.py: Made variance calculation consistent with point estimate
- var_probit now uses same 1/(2n) correction as safe_probit
- Was: hardcoded clip to [0.01, 0.99] regardless of sample size
- Now: uses 0.5/n and (n-0.5)/n bounds matching the point estimate
- Add _safe_binomial_loglik() using scipy's xlogy to correctly handle
edge cases where 0*log(0) should equal 0, not NaN. This fixes NaN
values in likelihood calculations when predicted probabilities are
exactly 0 or 1 (perfect discrimination scenarios).
- Fix NLL calculation in dprime_compare and posthoc pairwise tests to
use the new safe log-likelihood function, preventing NaN propagation
in test statistics.
- Rewrite _get_letters() CLD algorithm to use greedy clique cover on
the non-significance graph instead of incorrect graph coloring. Groups
sharing a letter must all be non-significantly different from each
other (form a clique in the non-significance graph). The algorithm now
iteratively finds maximal cliques to cover all non-significant pairs.
- Narrow exception handling in dod_power from bare "except Exception"
to "except (ValueError, RuntimeError, np.linalg.LinAlgError)" to
avoid masking unexpected errors during Monte Carlo simulation.
Implement Phase 2: DOD model, d-prime tests, and A-Not-A protocol
- Fix sdt() to handle edge cases that cause numerical issues:
- Add check for rows summing to zero with clear ValueError
- Implement Macmillan & Creelman (2005) 1/(2n) correction to clip
extreme proportions, preventing inf/-inf from ppf(0) or ppf(1)
- Fix logit transform to use clipped proportions avoiding div by zero
- Clean up plotting.py:
- Remove redundant `or True` condition in plot_sample_size_curve()
- Remove unused `make_subplots` import
- Remove unused `n_criteria` variable in sdt()
- Restore matplotlib as optional dependency with [static-plots] extra,
documenting the breaking change from removing it as a core dependency
- Add edge case tests for sdt():
- test_extreme_proportions: verifies no inf with 0/1 proportions
- test_zero_row_raises: verifies proper error for empty rows
- test_logit_extreme_proportions: verifies logit handles edge cases
- Fix docstrings for `scale` parameter in auc() and roc() functions:
- Was incorrectly documented as "noise to signal" ratio
- Corrected to "signal to noise" ratio to match R's sensR behavior
- Added clarification that scale=1 is equal-variance SDT
- Unify color definitions in plotting.py:
- Added COLOR_SEQUENCE constant for multi-series plots
- Replaced local colors list in plot_psychometric_comparison()
- Ensures consistent styling across all plot functions
plotting utils
Implements missing sensR features with full R validation:
Double Link Functions (senspy/links/double.py):
- double_twoafc_link() - Double 2-AFC protocol
- double_duotrio_link() - Double duo-trio protocol
- double_triangle_link() - Double triangle protocol
- double_threeafc_link() - Double 3-AFC protocol
- double_tetrad_link() - Double tetrad protocol
- get_double_link() - Factory function with flexible naming
Simulation Functions (senspy/simulation.py):
- discrim_sim() - Simulate replicated difference tests with optional
individual variability (overdispersion)
- samediff_sim() - Simulate same-different test data
Protocol Power Functions (senspy/protocol_power.py):
- samediff_power() - Simulation-based power for same-different tests
- twoac_power() - Exact power computation for 2-AC protocol using
likelihood root statistic
All functions validated against R sensR with golden value tests:
- Double links match to <1e-8 precision
- twoac_power matches to <1e-5 precision
- Golden values added to tests/fixtures/golden_sensr.json
Test coverage: 509 tests passing (added 69 new tests)
Add double link functions, simulation, and protocol power
Implements three major feature sets for sensPy:
1. Double Protocol Links (senspy/links/double.py)
- double_twoafc_link: Two independent 2-AFC trials (p_guess=0.25)
- double_duotrio_link: Two independent duo-trio trials (p_guess=0.25)
- double_triangle_link: Two independent triangle trials (p_guess=1/9)
- double_threeafc_link: Two independent 3-AFC trials (p_guess=1/9)
- double_tetrad_link: Two independent tetrad trials (p_guess=1/9)
- All validated against R sensR to <1e-8 precision
2. Simulation Functions (senspy/simulation.py)
- discrim_sim(): Simulate replicated difference tests with optional
overdispersion via individual d-prime variability
- samediff_sim(): Simulate same-different test data with tau/delta
parameters
3. Protocol-Specific Power (senspy/protocol_power.py)
- samediff_power(): Monte Carlo power for same-different tests
- twoac_power(): Exact power for 2-AC using signed likelihood root
statistic, validated against R sensR
Additional changes:
- Add golden validation tests against R sensR reference values
- Configure pytest-cov and Codecov integration for coverage reporting
- Export new functions from senspy.__init__
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add .gitattributes to enforce LF line endings for all text files, preventing future CRLF issues when working across platforms (Windows/WSL). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Cleanup: - Remove sensR/ directory (original R package source, no longer needed) - Remove .ai-docs/ (outdated AI-generated documentation) - Remove docs/GAP_ANALYSIS.md and docs/PORTING_PLAN.md (outdated planning docs) - Remove site/ (mkdocs build output) Update .gitignore: - Add site/, .pytest_cache/, .coverage, coverage.xml, htmlcov/, .venv/ 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Update development status classifier from Pre-Alpha to Beta - Add warning block to module docstring visible via help(senspy) - Add prominent disclaimer banner to README.md - Add Call for Testers section to README.md and CONTRIBUTING.md - Direct testers to info@aigora.com for team contact
There was a problem hiding this comment.
Pull request overview
This PR adds beta disclaimers to the package documentation, warning users that sensPy is currently in beta and results should be validated against sensR before use in production systems.
- Added a warning section to the main
__init__.pydocstring explaining the beta status - Updated the title to reference adding beta disclaimers for both Claude and xi.ai context
- Removed the R project file as this is now a Python-only repository
Reviewed changes
Copilot reviewed 111 out of 167 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| senspy/init.py | Added beta software warning to module docstring with contact information |
| sensR.Rproj | Deleted R project configuration file (no longer needed for Python package) |
| Multiple new Python files | Added new module implementations for links, dprime tests, DOD models, discrimination, core types, beta-binomial, A-Not-A, discrim, and various support files |
| Multiple .Rd files | Removed R documentation files (man/*.Rd) as package is now Python-based |
| scripts/generate_golden_data.R | Added R script to generate golden test data from sensR for validation |
| pyproject.toml | Added Python project configuration with Poetry |
| notebooks/tutorial.ipynb | Added tutorial notebook for package usage |
| mkdocs.yml | Added documentation site configuration |
| docs/ files | Added documentation markdown files for API reference and getting started guides |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Summary of ChangesHello @john-aigora, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request marks a significant transition, effectively migrating the entire Highlights
Ignored Files
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
| uses: actions/upload-pages-artifact@v3 | ||
| with: | ||
| path: site/ | ||
|
|
|
|
||
| - name: Run tests with coverage | ||
| run: uv run pytest tests/ -v --tb=short --cov=senspy --cov-report=xml --cov-report=term-missing | ||
|
|
|
|
||
| # See https://github.com/anthropics/claude-code-action/blob/main/docs/usage.md | ||
| # or https://code.claude.com/docs/en/cli-reference for available options | ||
| claude_args: '--allowed-tools "Bash(gh issue view:*),Bash(gh search:*),Bash(gh issue list:*),Bash(gh pr comment:*),Bash(gh pr diff:*),Bash(gh pr view:*),Bash(gh pr list:*)"' |
There was a problem hiding this comment.
Bug: Claude review workflow cannot post PR comments
claude-code-review.yml grants only pull-requests: read, but the provided prompt instructs using gh pr comment to leave feedback. Without pull-requests: write (or issues: write if commenting via issue comments), the workflow won’t be able to publish the review comment.
| "Bash(~/.local/bin/uv pip install:*)" | ||
| ] | ||
| } | ||
| } |
There was a problem hiding this comment.
There was a problem hiding this comment.
Code Review
This pull request represents a significant effort to port the sensR package from R to a new Python package, sensPy. The changes are comprehensive, including the removal of old R files, the addition of a complete Python project structure with pyproject.toml, extensive documentation using mkdocs, and the implementation of the core library. The code is well-structured and follows modern Python practices. My review has identified a high-severity security concern regarding overly permissive shell commands in a configuration file. Additionally, I've found several incorrect code examples in the README, testing strategy, and tutorial notebook that could mislead users. Addressing these issues will greatly improve the project's security and usability.
| { | ||
| "permissions": { | ||
| "allow": [ | ||
| "Bash(rm:*)", |
There was a problem hiding this comment.
The permission Bash(rm:*) is overly broad and presents a significant security risk. It allows the execution of rm with any arguments, which could lead to the unintentional deletion of any file or directory. It's crucial to restrict this permission to only what is absolutely necessary.
"Bash(rm:-rf:build dist *.egg-info site .pytest_cache .coverage coverage.xml htmlcov)"| # Analyze a Triangle test result (80 correct out of 100) | ||
| result = discrim(correct=80, total=100, method="triangle") | ||
| print(f"d-prime: {result.d_prime:.3f}") | ||
| print(f"95% CI: [{result.ci_lower:.3f}, {result.ci_upper:.3f}]") |
There was a problem hiding this comment.
The example for accessing the confidence interval is incorrect. The DiscrimResult object does not have ci_lower and ci_upper attributes. The correct way to retrieve the confidence interval is by calling the confint() method, which returns a NumPy array containing the lower and upper bounds.
| print(f"95% CI: [{result.ci_lower:.3f}, {result.ci_upper:.3f}]") | |
| ci = result.confint() | |
| print(f"95% CI: [{ci[0]:.3f}, {ci[1]:.3f}]") |
| result = dprime_compare( | ||
| correct=[80, 65, 90], | ||
| total=[100, 100, 100], | ||
| method="triangle" | ||
| ) | ||
| print(f"Chi-square: {result.statistic:.2f}, p={result.p_value:.4f}") |
There was a problem hiding this comment.
This example for dprime_compare has two issues:
- The function expects a
protocolargument, notmethod. - The result object has a
stat_valueattribute for the test statistic, notstatistic.
I've corrected both in the suggestion.
| result = dprime_compare( | |
| correct=[80, 65, 90], | |
| total=[100, 100, 100], | |
| method="triangle" | |
| ) | |
| print(f"Chi-square: {result.statistic:.2f}, p={result.p_value:.4f}") | |
| result = dprime_compare( | |
| correct=[80, 65, 90], | |
| total=[100, 100, 100], | |
| protocol=["triangle", "triangle", "triangle"] | |
| ) | |
| print(f"Chi-square: {result.stat_value:.2f}, p={result.p_value:.4f}") |
| "pc": {"estimate": coef[0, 0], "std_err": coef[0, 1]}, | ||
| "pd": {"estimate": coef[1, 0], "std_err": coef[1, 1]}, | ||
| "d_prime": {"estimate": coef[2, 0], "std_err": coef[2, 1]}, | ||
| "loglik": float(result.rx2("logLik")[0]), |
There was a problem hiding this comment.
The example code for the rpy2_helper attempts to access result.rx2("logLik"). However, the discrim function in sensR does not return a logLik element at the top level of its result object. This would likely cause an error in the data generation script. The log-likelihood might be available within a nested profile object if statistic="likelihood" is used, but it's not directly accessible as shown.
| "Compare d-prime across multiple groups." | ||
| ] | ||
| }, | ||
| { |
There was a problem hiding this comment.
The example code for betabin is incorrect. The BetaBinomialResult object returned by sp.betabin does not have a d_prime attribute directly. You need to call the summary() method on the result object first, which returns a summary object containing the d_prime estimate.
"summary = result.summary()\n",
"print(f\"D-prime: {summary.estimates['d_prime']:.3f}\")\n",
Note
Adds a new Python package (sensPy) implementing Thurstonian sensory discrimination models, power/sample size, ROC/plotting, and extensive tests/docs, including golden-data validation against sensR.
links/psychometric, double-protocol links).discrim,twoac,samediff,dod,anota,roc(SDT/ROC/AUC),betabin.utils), power/sample size (power,protocol_power), and simulations (simulation).tests/fixtures/golden_sensr.jsonand generator scriptscripts/generate_golden_data.R.mkdocs.yml) and tutorial notebook (notebooks/tutorial.ipynb).pyproject.toml(dependencies, tooling); adduv.lock.man/twofive.Rd,man/twofiveF.Rd) and legacy R tests/vignettes.Written by Cursor Bugbot for commit 1df59e2. This will update automatically on new commits. Configure here.