For maintainers

You already maintain this collector. Get paid for it.

If you have built a scraper for a source you understand better than almost anyone — an ISO feed, a tariff book, a queue report — you have probably rebuilt it three times as the source changed, and probably only ever been paid for it once.

Being straight about where this is: there is no self-serve publishing yet, and no maintainers besides us. We are writing the first two collectors ourselves to prove the pipeline works before asking anyone else to depend on it. If the model appeals, talk to us now and you will shape the terms rather than inherit them.

Talk to us

What a collector is

Where the data comes from, why you may redistribute it, how fresh it has to be, what shape it emits, and what counts as correct. Then one method.

collectors/ercot_spp.py
class ErcotDayAheadSpp(Collector):
    spec = CollectorSpec(
        slug="ercot-day-ahead-spp",
        source_url=ERCOT_API,
        basis=SourceBasis.ISO_PUBLIC,

        schedule="30 20 * * *",
        freshness_sla=timedelta(hours=30),

        schema=LMP_SCHEMA,
        thresholds=Thresholds(
            null_rate={"lmp_energy": 1.0},
            row_count_tolerance=0.15,
            value_bands={"lmp_total": (-2_000.0, 6_000.0)},
        ),
    )

    async def run(self, ctx: RunContext) -> pd.DataFrame:
        resp = await ctx.fetch(ERCOT_API, params=..., retries=3)
        return normalise(resp.json()["data"])

Note what run does not do: it returns a frame rather than writing anywhere. Batches are staged, validated and only then promoted, so a collector cannot put bad data in front of a buyer even by accident.

What we ask of you

Short list, but we hold you to it — the governance claim is the whole product.

  1. 1
    Declare a freshness SLA you can actually hit

    Looser than your cadence, so one missed run is a retry. Two consecutive misses flag the listing publicly and start a grace period.

  2. 2
    Declare thresholds that mean something

    Null budgets, row-count tolerance, value bands. Set them where a real break trips them — thresholds so loose nothing ever fails are worse than none, because the badge stays green while the data rots.

  3. 3
    Say why you may redistribute it

    You state the legal basis, we review it before the listing goes live, and we will pull a dataset over this.

  4. 4
    Answer when it breaks

    You will know first — validation fails the batch before any buyer sees it. What buyers are actually paying for is that you respond.

Revenue share, payout cadence and minimums are not settled. We would rather agree them with the first maintainers than announce terms and defend them.