Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -105,3 +105,6 @@ BACKUP_VPS_PASSWORD=
BACKUP_VPS_KEY_PATH=
BACKUP_VPS_PORT=22
BACKUP_VPS_ROOT=/backups

GITHUB_API_TOKEN=
GITHUB_WEBHOOK_SECRET=
18 changes: 10 additions & 8 deletions CONTEXT-MAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,15 @@ This is a modular monorepo (`internachi/modular`). Each bounded context lives un

## Contexts

| Context | Path | Description |
| ------------------- | ---------------------------------- | --------------------------------------------------------------------------- |
| Moderation | `app-modules/moderation/` | Content moderation pipeline — classification, routing, enforcement, appeals |
| Bot Discord | `app-modules/bot-discord/` | Discord bot runtime (Laracord websocket, slash commands, event handlers) |
| Integration Discord | `app-modules/integration-discord/` | Discord platform transport (REST API via Saloon), OAuth, ETL |
| Identity | `app-modules/identity/` | Users, tenants, external identities, authentication |
| Panel Admin | `app-modules/panel-admin/` | Filament admin panel — dashboards, resources, moderation UI, marketing |
| Integration Twitch | `app-modules/integration-twitch/` | Twitch platform transport (Helix API via Saloon), OAuth, EventSub webhooks |
| Context | Path | Description |
| ------------------- | ---------------------------------- | -------------------------------------------------------------------------------------------------------------- |
| Moderation | `app-modules/moderation/` | Content moderation pipeline — classification, routing, enforcement, appeals |
| Bot Discord | `app-modules/bot-discord/` | Discord bot runtime (Laracord websocket, slash commands, event handlers) |
| Integration Discord | `app-modules/integration-discord/` | Discord platform transport (REST API via Saloon), OAuth, ETL |
| Identity | `app-modules/identity/` | Users, tenants, external identities, authentication |
| Panel Admin | `app-modules/panel-admin/` | Filament admin panel — dashboards, resources, moderation UI, marketing |
| Integration Twitch | `app-modules/integration-twitch/` | Twitch platform transport (Helix API via Saloon), OAuth, EventSub webhooks |
| Integration GitHub | `app-modules/integration-github/` | GitHub transport (REST via Saloon), OAuth, community contribution ingestion (backfill + webhooks) + event lake |

## Relationships

Expand Down Expand Up @@ -41,4 +42,5 @@ This is a modular monorepo (`internachi/modular`). Each bounded context lives un
- **Bot Discord** depends on Moderation (listens to domain events) and Integration Discord (uses transport).
- **Integration Discord** depends on Identity (OAuth user resolution). It never imports from Moderation.
- **Integration Twitch** depends on Identity (OAuth user resolution, ExternalIdentity for tenant linking). It never imports from Moderation, Integration Discord, or Bot Discord.
- **Integration GitHub** depends on Identity (OAuth user resolution; future `Character` seam via `ExternalIdentity`). It never imports from Activity, Economy, Moderation or any Bot/runtime module — it only emits the `GithubContributionRecorded` domain event. The community presentation (in `portal`) and the allowlist admin UI (in `panel-admin`) depend on it, never the reverse.
- **Identity** has no upstream dependencies on other contexts listed here.
53 changes: 53 additions & 0 deletions app-modules/integration-github/CONTEXT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Integration GitHub Context

Transport and integration layer for the GitHub platform. Owns all HTTP communication with GitHub's REST API (via Saloon), OAuth flows, the ingestion of community contributions (backfill + webhooks), and the raw event lake.

## Glossary

| Term | Definition | Not to be confused with |
| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------- |
| **Transport** | The Saloon-based HTTP layer (`Transport/`) that sends requests to GitHub's REST API. All GitHub HTTP goes through here, authenticated with a PAT. | The OAuth connector (which exchanges user login codes) |
| **Contribution** | A normalized record of one thing a person did on a tracked repo: a PR, review, issue, comment or commit. Keyed by GitHub login — member or not. | `Interaction` (the gamification record in `activity`, members-only) |
| **Allowlist** | The set of repos that count, stored in `github_repositories` (one row per tenant + repo) and managed in the admin panel. Ingestion is scoped to it and fans out per tenant tracking the repo. | All org repos (the org webhook delivers everything; we filter) |
| **Event lake** | `github_event_logs` — the raw, append-only store of webhook payloads (deduped by delivery id), for audit and replay. | `github_contributions` (the normalized read model) |
| **Backfill** | Historical import via paginated REST list requests, upserting contributions directly. Resumable via `last_backfilled_at`. | Webhook ingestion (the live stream through the lake) |

## Structure

```
src/
├── Transport/
│ ├── GitHubApiConnector.php ← REST base URL + PAT auth (services.github.api_token)
│ ├── GitHubOAuthConnector.php ← OAuth client credentials
│ └── Requests/
│ ├── Users/ ← GetCurrentUser, GetUser
│ └── Contributions/ ← ListPullRequests, GetPullRequest, ListIssues,
│ ListReviews, ListIssueComments, ListPrComments, ListCommits
├── OAuth/ ← GitHubOAuthClient (implements OAuthClientContract)
├── Backfill/ ← BackfillRepository action + console command
├── Webhook/ ← VerifyGithubSignature, GithubWebhookController, ProjectGithubEvent
├── Models/ ← GithubRepository · GithubContribution · GithubEventLog
└── Events/ ← GithubContributionRecorded (seam for gamification)
```

## Module Boundaries

### This module owns:

- All HTTP requests to GitHub REST API (PAT + OAuth)
- OAuth token exchange and user profile retrieval
- The community contribution model and its ingestion (backfill + webhook)
- The raw GitHub event lake

### This module does NOT own:

- Gamification of contributions (coins/xp) — belongs to `activity`/`economy`; this module only **emits** `GithubContributionRecorded`
- The community presentation page — belongs to `portal` (reads `github_contributions`)
- The admin UI — the Filament resources live in `panel-admin`

## Dependencies

- **Identity** — OAuth user resolution (`OAuthClientContract`, `OAuthUserDTO`); and, as a future seam, resolving a `Character` from a GitHub `ExternalIdentity`.
- **No dependency on** Activity, Economy, Moderation or any Bot/runtime module.

See `docs/adr/0001-github-community-contributions.md` for the decisions behind this design.
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
<?php

declare(strict_types=1);

namespace He4rt\IntegrationGithub\Database\Factories;

use He4rt\Identity\Tenant\Models\Tenant;
use He4rt\IntegrationGithub\Enums\ContributionType;
use He4rt\IntegrationGithub\Models\GithubContribution;
use Illuminate\Database\Eloquent\Factories\Factory;

/**
* @extends Factory<GithubContribution>
*/
final class GithubContributionFactory extends Factory
{
protected $model = GithubContribution::class;

/**
* @return array<string, mixed>
*/
public function definition(): array
{
$number = fake()->unique()->numberBetween(1, 1_000_000);

return [
'tenant_id' => Tenant::factory(),
'repo' => 'he4rt/heartdevs.com',
'actor_login' => fake()->userName(),
'actor_id' => fake()->numberBetween(1, 9_999_999),
'type' => ContributionType::Pr,
'external_ref' => 'pr:'.$number,
'target_ref' => null,
'occurred_at' => now(),
'metadata' => [],
];
}

public function bot(): self
{
return $this->state(fn (array $attributes): array => [
'actor_login' => 'dependabot[bot]',
'metadata' => [...$attributes['metadata'] ?? [], 'is_bot' => true],
]);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
<?php

declare(strict_types=1);

namespace He4rt\IntegrationGithub\Database\Factories;

use He4rt\Identity\Tenant\Models\Tenant;
use He4rt\IntegrationGithub\Models\GithubRepository;
use Illuminate\Database\Eloquent\Factories\Factory;

/**
* @extends Factory<GithubRepository>
*/
final class GithubRepositoryFactory extends Factory
{
protected $model = GithubRepository::class;

/**
* @return array<string, mixed>
*/
public function definition(): array
{
return [
'tenant_id' => Tenant::factory(),
'full_name' => 'he4rt/'.fake()->unique()->slug(2),
'enabled' => true,
'last_backfilled_at' => null,
];
}

public function disabled(): self
{
return $this->state(['enabled' => false]);
}

public function backfilled(): self
{
return $this->state(['last_backfilled_at' => now()]);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
<?php

declare(strict_types=1);

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

return new class extends Migration
{
public function up(): void
{
Schema::create('github_repositories', function (Blueprint $table): void {
$table->uuid('id')->primary();
$table->foreignUuid('tenant_id')->constrained('tenants');
$table->string('full_name'); // owner/repo
$table->boolean('enabled')->default(true);
$table->timestamp('last_backfilled_at')->nullable();
$table->timestamps();

// A allowlist é por tenant: cada comunidade mantém a sua, e o mesmo
// repo público pode ser acompanhado por mais de uma comunidade.
$table->unique(['tenant_id', 'full_name'], 'uniq_github_repositories_tenant_repo');
});
}

public function down(): void
{
Schema::dropIfExists('github_repositories');
}
};
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
<?php

declare(strict_types=1);

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

return new class extends Migration
{
public function up(): void
{
Schema::create('github_contributions', function (Blueprint $table): void {
$table->uuid('id')->primary();
$table->foreignUuid('tenant_id')->constrained('tenants');
$table->string('repo');
$table->string('actor_login');
$table->unsignedBigInteger('actor_id')->nullable();
$table->string('type');
$table->string('external_ref');
$table->string('target_ref')->nullable();
$table->timestamp('occurred_at');
$table->jsonb('metadata')->nullable();
$table->timestamps();

// Isolamento por tenant: a mesma contribuição de um repo compartilhado
// é gravada uma vez por comunidade que acompanha o repo.
$table->unique(['tenant_id', 'repo', 'type', 'external_ref'], 'uniq_github_contributions_ref');
$table->index(['tenant_id', 'occurred_at'], 'idx_github_contributions_tenant_time');
$table->index('actor_id', 'idx_github_contributions_actor');
$table->index(['tenant_id', 'type', 'occurred_at'], 'idx_github_contributions_type_time');
});
}

public function down(): void
{
Schema::dropIfExists('github_contributions');
}
};
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
<?php

declare(strict_types=1);

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

return new class extends Migration
{
public function up(): void
{
Schema::create('github_event_logs', function (Blueprint $table): void {
$table->id();
$table->string('event_type')->index();
$table->string('repo')->nullable()->index();
$table->string('actor_login')->nullable();
$table->string('delivery_id')->nullable()->unique();
$table->jsonb('payload');
$table->timestamps();
});
}

public function down(): void
{
Schema::dropIfExists('github_event_logs');
}
};
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# ADR-0001: GitHub Community Contributions

**Status:** Accepted
**Date:** 2026-06-04
**Deciders:** Clintonrocha98

## Context

The community wanted a recurring presentation ("Quem fez a He4rt bater") showing who is contributing across the community's public GitHub repos. The first attempt was a provisional artisan command (`community:retrospective`) that ran `gh` live, aggregated a JSON in-memory and uploaded it to a paste host. It did not persist anything, ran outside the modular architecture (under `app/Retrospective/`), and only worked for a single repo at a time.

We need a persistent, queryable model that:

- Counts **all** contributors by GitHub login — whether or not they are registered He4rt members.
- Captures **history** (backfill) and **new activity** (live).
- Spans **multiple repos**, curated and maintainable without code changes.
- Lives inside the modular architecture and respects boundaries.

The existing `activity` domain has an `Interaction` model, but it is gamification-coupled: it requires a `character_id` (a registered member) and a `tenant_id`, and carries coins/xp. It cannot represent a GitHub contributor who never joined the platform.

## Decision

Build the ingestion in `integration-github` (transport), mirroring the Discord data-lake pattern.

### 1. Own contribution model, not `Interaction`

A normalized `github_contributions` table keyed by GitHub login/id (member-or-not), with `unique(tenant_id, repo, type, external_ref)` for idempotency. Gamification is left as a **seam**: this module emits `GithubContributionRecorded` and does not depend on `activity`/`economy`.

### 2. Normalized read model + raw lake

`github_contributions` is the read model the presentation queries. `github_event_logs` is the raw, append-only webhook lake (deduped by delivery id) for audit/replay — exactly mirroring `discord_event_logs`.

### 3. Split ingestion paths

- **Webhook (live):** org webhook → signature-verified → raw payload into the lake → `ProjectGithubEvent` **fans out**, projecting one contribution per tenant whose allowlist enables the repo.
- **Backfill (history):** paginated REST list requests upsert contributions directly, stamping the source repo's `tenant_id`. Resumable via `last_backfilled_at`. The `unique(tenant_id, repo, type, external_ref)` key makes both paths converge without duplicates.

### 4. Panel-managed allowlist, scoped per tenant

`github_repositories` (managed in `panel-admin`) is the source of truth for which repos count. **Each repo belongs to a tenant** (`unique(tenant_id, full_name)`): the allowlist is per community, and the same public repo may be tracked by more than one tenant. The org webhook delivers everything; ingestion filters by the allowlist and fans out to every tenant tracking the repo.

### 4b. Tenant isolation

Both `github_repositories` and `github_contributions` carry `tenant_id`; the presentation, the panel resource and the live/backfill writers are all scoped by it. The raw lake (`github_event_logs`) stays **global** — one delivery is stored once and projected to N tenants. Because public GitHub data is identical across communities, a shared repo's contributions are deliberately **duplicated per tenant** (one row each) to keep every community's read model fully isolated and independently queryable. The public retrospective resolves the tenant from a route slug (`/comunidade/{tenant}/retrospectiva`), defaulting to `config('he4rt.main_tenant')` when none is given.

### 5. Store everything, filter on read

Everything is stored raw. The presentation excludes only **bots** at read time. **Closed-unmerged PRs count as participation** (consistent with the "all contributors" audience), but are broken out by outcome (`prs_merged` / `prs_unmerged`) so the view distinguishes work that landed from work that was rejected/abandoned. Because filtering lives only in the read model, this refinement was a one-function change with no re-backfill — exactly the payoff of storing everything raw.

### 6. Auth and scope

A fine-grained PAT (`services.github.api_token`) authenticates the REST connector for backfill; a manually-registered org webhook with a shared secret (`services.github.webhook_secret`) feeds the live stream. Contribution types in scope: PR, review, issue, comment, commit (no reactions — GitHub has no reaction webhook).

## Consequences

- **Positive:** the presentation is fast and queryable by period/person/type; history and live converge idempotently; gamification can be wired later without re-modeling; adding a repo is a panel action; each community's data is fully isolated and consistent with the rest of the multi-tenant panel.
- **Negative:** PR size enrichment costs one extra request per PR (the dominant cost — ~2 requests/PR); reactions are out of scope; the org webhook delivers traffic for non-allowlisted repos (stored in the lake for audit, then ignored); a repo shared by N tenants stores its contributions N times (accepted: public data is small and isolation is worth more than the dedup).
- **Backfill scale & resilience:** the backfill is **synchronous and rate-limit-aware** — every request uses Saloon `->throw()`, so a 403/5xx fails loudly (no silent corruption); on a rate-limit 403 both backfill transports (the `github:backfill` console command and the panel's "Backfill agora" action) surface a clear, actionable message and is safely resumable (idempotent upsert means a re-run after the reset costs nothing extra and never duplicates). The rate-limit interpretation lives in a single shared `Backfill\RateLimit` helper so the two transports cannot drift. At current scale this is enough: the main repo (~230 PRs) is ~500 requests, ~10% of the 5,000/hour budget. A **queue** (job-per-repo with release-on-limit) and a true **incremental** refresh (using `last_backfilled_at` as `since` — today it is recorded but each run still does full history) are deliberately deferred as YAGNI; revisit if many large repos are added.
- **Follow-up:** revisit a lake retention/TTL policy; decide whether commits without a linked GitHub user enter the per-person ranking; implement the gamification bridge via the seam event.
13 changes: 13 additions & 0 deletions app-modules/integration-github/routes/github-webhook-routes.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<?php

declare(strict_types=1);

use He4rt\IntegrationGithub\Webhook\GithubWebhookController;
use He4rt\IntegrationGithub\Webhook\VerifyGithubSignature;
use Illuminate\Support\Facades\Route;

Route::prefix('api/webhooks/github')
->middleware([VerifyGithubSignature::class])
->group(function (): void {
Route::post('/', GithubWebhookController::class)->name('github.webhook');
});
Loading