Skip to content

feat: add normalizers for Alpha Vantage etc#25

Merged
Villarley merged 7 commits intoNeko-Protocol:mainfrom
davedumto:feat/data-normalization
Jan 31, 2026
Merged

feat: add normalizers for Alpha Vantage etc#25
Villarley merged 7 commits intoNeko-Protocol:mainfrom
davedumto:feat/data-normalization

Conversation

@davedumto
Copy link
Contributor

🚀 Neko Pull Request

Mark with an x all the checkboxes that apply (like [x])

  • Closes Data normalization #17
  • Added tests (if necessary)
  • Run tests
  • Run formatting
  • Evidence attached
  • Commented the code

📌 Type of Change

  • Documentation (updates to README, docs, or comments)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

📝 Changes description

Implemented a Data Normalization Service for the aggregator app that converts raw price data from multiple external sources into a standardized internal format.

Features

  • NormalizedPrice Interface: Standard format with symbol, price (4 decimals), ISO 8601 timestamp, and metadata
  • Strategy Pattern: Extensible normalizer architecture with source-specific implementations
  • Source Support: Alpha Vantage, Finnhub, Yahoo Finance, and Mock providers
  • Transformations:
    • Ticker symbols normalized (e.g., AAPL.USAAPL, US-GOOGLGOOGL)
    • Prices rounded to 4 decimal places
    • Timestamps converted to ISO 8601 UTC
  • Error Handling: ValidationException for malformed data with descriptive messages
  • Metadata Tracking: Original source, symbol, and list of transformations applied

Files Added

  • apps/aggregator/src/interfaces/normalized-price.interface.ts
  • apps/aggregator/src/interfaces/normalizer.interface.ts
  • apps/aggregator/src/normalizers/base.normalizer.ts
  • apps/aggregator/src/normalizers/alpha-vantage.normalizer.ts
  • apps/aggregator/src/normalizers/finnhub.normalizer.ts
  • apps/aggregator/src/normalizers/yahoo-finance.normalizer.ts
  • apps/aggregator/src/normalizers/mock.normalizer.ts
  • apps/aggregator/src/services/normalization.service.ts
  • apps/aggregator/src/services/normalization.service.spec.ts
  • apps/aggregator/src/exceptions/normalization.exception.ts
  • apps/aggregator/src/modules/normalization.module.ts

Test Results

  • 134 tests passed
  • 86.63% code coverage (exceeds >85% requirement)

📸 Evidence (A Loom/Cap video is required as evidence, we WON'T merge if there's no proof)

Screenshot 2026-01-28 at 9 53 57 PM

⏰ Time spent breakdown

Task Time
Planning & Architecture 30 min
Interfaces & Types 15 min
Normalizer Implementations 45 min
NormalizationService 20 min
Unit Tests 40 min
Documentation 15 min
Total ~2.5 hours

🌌 Comments

  • Used Strategy pattern for easy extensibility - adding new data sources only requires implementing the Normalizer interface
  • All normalizers share common logic via BaseNormalizer abstract class
  • Metadata tracking enables full audit trail of transformations

Thank you for contributing to Neko, you just helped us make RWAs consumer friendly on Stellar! We hope you can continue contributing to this project.

davedumto and others added 2 commits January 28, 2026 21:14
…Mock data sources

- Implemented AlphaVantageNormalizer to handle exchange suffixes and case variations.
- Created FinnhubNormalizer to manage exchange prefix formats.
- Developed YahooFinanceNormalizer to process various exchange suffixes and index prefixes.
- Introduced MockNormalizer for testing and development purposes.
- Established a base normalizer class to provide common functionality across all normalizers.
- Added unit tests for each normalizer to ensure correct behavior and transformations.
- Implemented NormalizationService to manage normalization processes and register normalizers.
- Enhanced error handling and validation for raw price data.
Copy link
Member

@Villarley Villarley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

…ation

- Separate NormalizedPriceRecord (normalization output) from NormalizedPrice (aggregation input)
- Update all normalizers and services to use correct interface types
- Fix lint errors from merge conflicts
@davedumto
Copy link
Contributor Author

hi @Villarley please merge

Copy link
Member

@Villarley Villarley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Villarley Villarley merged commit 329da0d into Neko-Protocol:main Jan 31, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Data normalization

2 participants