Field guide to the data

The model in one paragraph

A parcel is not a meter. This dataset exists to get from the thing counties publish — a polygon with an assessor's use code — to the thing SCE dispatches crews against: a metered service connection sitting on one line section, in one census block, with a defensible statement of how confidently we know both.

Every number here is therefore two numbers: an estimate and a label saying how it was derived. The labels are measured, inferred and modelled, they are reported separately, and they are never averaged. A meter total without its method mix is not a usable figure in this project.

Two files carry every judgement call, and neither is code: config/crosswalk.csv says what a parcel is, and config/meters.yml says how many electric services its dwelling units imply. They are kept apart on purpose — a reviewer can accept the classification and still argue with the conversion, which is the argument worth having.

Where it came from

Thirty sources, all harvested live and all ArcGIS REST, totalling 8,637,423 assembled features with zero count drift against the register. Every harvested row carries source_url and retrieved_at; every harvest writes a companion *.manifest.json recording what the service claimed to hold versus what actually arrived. Provenance is not an afterthought in a bid deliverable, and a hand-maintained source list would already be wrong.

SCE's own grid geography

The most important design decision in the project: SCE publishes its network, so results roll up along its geography rather than ours.

County parcels — all fifteen counties

Fresno, Kings, Madera and Inyo publish no parcel service and are resolved by direct assessor records request, not by scraping third-party mirrors: provenance that cannot be cited is worthless in an RFP. All four are minority-share SCE territory, so they are the cheapest to lose.

Counties are graded by distance from the published layer to a per-parcel meter estimate, not by data quality in the abstract:

• Tier A — dwelling-unit counts published directly. Los Angeles, Riverside.
• Tier B — a use code or zoning string is present; units must be modelled from it.
• Tier C — geometry and an assessor number only. Needs an assessor-roll join before it can be classed at all.
• Tier D — no public service.

Census geography — the second axis, and the validation control

270,639 blocks, 482 places and 1,129 CDPs across the fifteen counties, all matching expected counts exactly. Only blocks are pulled: the 15-character block GEOID nests exactly, so block group is the first 12 characters, tract the first 11 and county the first 5. One spatial join yields three levels by string truncation, and truncation cannot disagree with itself the way three separate joins can.

TIGERweb's block layer carries HU100 and POP100 itself, so the 2020 decennial housing-unit control arrives with the geometry — the independent per-block check the calibration gate needs, with no separate Census API pull. Across all fifteen counties, unclipped: 7,750,632 housing units and 22,060,189 people. That is the frame the 5.7M meter figure sits inside, not a number to compare it against; roughly a third of those counties belongs to a utility other than SCE.

How a county endpoint becomes a scored line section

Why the territory carve-out is opt-in, not a blanket difference

SCE's own polygon already excludes the large historic municipal holes — Glendale and Burbank do not overlap it at all, Pasadena 0.9%, Anaheim 5.7%. What it does not remove is a shorter list of newer publicly owned utilities sitting entirely inside it: Moreno Valley, City of Industry, Cerritos, Victorville, Corona, Rancho Cucamonga, plus Needles and Bear Valley Electric.

Two aggregation axes, and why neither derives the other

Meters must roll up both along SCE's network and along standard census geography. They answer different questions and neither substitutes for the other: the grid axis prices deployment and phases the work, the census axis supplies the independent housing-unit control and the equity reporting the CPUC requires.

The two ambiguities are different problems

The files on disk, and what each column means

Everything is per-county Parquet under data/core/, in three families that join on parcel_uid. Geometry is GeoParquet 1.0 WKB in EPSG:3310, California Albers — all area, length and distance maths happens there, and reprojection to 4326 or 3857 happens only at export. Areas computed in Web Mercator are wrong and this project never does it.

<county>_sce.parquet — clipped parcels

The source layer's own fields, lowercased and otherwise untouched, plus the keys the project adds. Raw use codes and zoning strings are preserved forever alongside normalised values: every reviewer will want to audit a crosswalk decision.

<county>_parcel_alloc.parquet — stage 1, where the parcel is

One row per (parcel, block), not per parcel. Complete for all fifteen counties: 4,133,590 rows, zero parcels failed to find a line section.

<county>_units.parquet — stage 2, how many meters it holds

One row per parcel. Currently built for Los Angeles, Riverside parcels and Riverside condominium polygons.

The conversion from units to meters

meters = dwelling_units × per_unit × (1 − master_metered_share) + per_parcel

Every constant in that formula lives in config/meters.yml with a written rationale, because none of them can be verified from public data. Two of the rows exist specifically to avoid a trap the assessor data walks you into: hotels and institutions report rooms and beds as “units”, so both carry per_unit: 0. Los Angeles alone reports 52,782 hotel “units” across 1,270 parcels; a 200-room hotel is one service, not 200. Deleting that one line inflates the county by fifty thousand meters.

How to query it

Nothing needs to be online. The rollups are non-spatial joins over Parquet — the spatial work is already done and cached in stage 1 — so DuckDB against the files is the fastest route to an answer.

Meters per line section, rolled up to circuit

# the canonical rollup: weight by block_share, never sum from a modal key
import duckdb
duckdb.sql("""
  SELECT a.circuit_name,
         a.substation_name,
         sum(u.meters_est * a.block_share)                              AS meters,
         sum(u.meters_est * a.block_share)
           FILTER (WHERE u.meters_method = 'measured')                  AS measured,
         count(DISTINCT a.section_id)                                   AS sections
  FROM   'data/core/la_parcels_parcel_alloc.parquet' a
  JOIN   'data/core/la_parcels_units.parquet'        u USING (parcel_uid)
  GROUP  BY 1, 2
  ORDER  BY meters DESC
""").show()

The same meters on the census axis, with the independent control

# three geography levels from one column, by truncation
duckdb.sql("""
  SELECT substr(a.block_geoid, 1, 11)                AS tract_geoid,
         sum(u.meters_est * a.block_share)           AS meters,
         sum(u.dwelling_units * a.block_share)       AS units
  FROM   'data/core/la_parcels_parcel_alloc.parquet' a
  JOIN   'data/core/la_parcels_units.parquet'        u USING (parcel_uid)
  GROUP  BY 1
""").show()

Geometry, when you actually need it

# GeoParquet 1.0, EPSG:3310 — geopandas reads the CRS from the file
import geopandas as gpd
pts = gpd.read_parquet("data/core/la_parcels_parcel_alloc.parquet")   # points
poly = gpd.read_parquet("data/core/la_parcels_sce.parquet")           # polygons

Four rules that keep the answer right

Published surfaces

Anything baked into a tile is filterable client-side with nothing running behind the map; anything omitted needs a database. config/tiles.yml keeps the field allowlist deliberately short for that reason, and excludes provenance columns — they are per-layer constants and belong in the archive description, not repeated on every feature in every tile.