Fall back to old bhavcopy format (with ISIN) when udiff unavailable - #135
Merged
Conversation
bhavcopy_raw previously fell back to the full bhavcopy (sec_bhavdata_full,
no ISIN column) when UDiff sources failed. Now it falls back to the legacy
cm{dd}{MMM}{yyyy}bhav.csv.zip file, fetched through the NSE reports API:
/api/reports?archives=[{"name": "CM - Bhavcopy(csv)", "type":
"daily-reports", "category": "capital-market", "section": "equities"}]
&date={DD-MMM-YYYY}&type=equities&mode=single
- Add NSEArchives.bhavcopy_old_raw (exported as module-level bhavcopy_old_raw)
- Handles 403 cookie bootstrap and raises for missing files (404)
- Update test_bhavcopy to expect old format with ISIN for historical dates
- Mark network tests in tests/test_bhav.py as live; add mocked offline
fallback tests for CI
- Bump version to 0.35.5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
bhavcopy_rawnow falls back to the old bhavcopy format (legacycm{dd}{MMM}{yyyy}bhav.csv.zip, which includes the ISIN field) instead of the full bhavcopy (sec_bhavdata_full, which has no ISIN).NSEArchives.bhavcopy_old_raw()(also exported as module-levelbhavcopy_old_raw) which fetches the file through the NSE reports API:requests.RequestExceptionwhen NSE returns 404 (file unavailable, e.g. UDiff-era dates).Fallback order now
Tests
test_bhavcopy(live): historical date 2020-01-01 must return the old format with ISIN header and must NOT containDELIV_QTYcolumns.bhavcopy_old_rawdownload incl. ISIN value check; 404 for post-Jul-2024 dates.full_bhavcopy_raw; pre-UDiff dates skip udiff entirely; correct reports-API URL/params + unzip; 404 raises.tests/test_bhav.pyas@pytest.mark.liveper repo convention, keeping CI green without network.Misc
pytest>=8.0(pytest 7.1.2 is incompatible with Python 3.14 —ast.Strremoved)Verified live: full suite passes (73 passed), and
bhavcopy_save(date(2020,1,1))savescm01Jan2020bhav.csvwith ISIN column populated.Supersedes #134 (re-created under correct account with re-authored commit).