# Synthetic Forestry Dataset - Maine & Northern New England

**Preliminary release 1.0 - ENTIRELY FICTITIOUS**

Published by Krummholz AI for data-cleaning research and evaluation.
Records are generated procedurally with seed 20240617; reference tables and
fictional field notes are embedded in the generator. No real observations
or external datasets are loaded. Geographic names and approximate bounds
provide regional context, not verified plot locations.

This preliminary set is not production-ready or scientifically validated.
Do not use it to prescribe forestry treatments, estimate real inventories,
or make operational, ecological, land-management or safety decisions.
Species codes and growth/volume relationships are illustrative.

The intentional missing values, duplicates, impossible measurements,
inconsistent dates/units/categories, malformed CSV rows and nested JSON
defects are retained. The defect manifest is a partial record of injections,
not comprehensive ground truth; some structural defects are not logged and
soil-profile defect indexing is imprecise. The loader is a demonstration,
not a complete validator or trustworthy data-cleaning pipeline.

## What this is

A synthetic, plot- and tree-level forest inventory dataset styled after
field data collected in Maine, New Hampshire and Vermont. It was generated
procedurally from a fixed random seed. **No real plot, landowner, crew or
tree measurement is represented here.** Every identifier and value is
invented.

## Why it contains defects

Real forestry data is messy. This dataset intentionally includes missing
values, duplicates, impossible numbers, inconsistent categorical strings,
mixed date formats, unit errors and structurally broken rows so that it can
be used to train or evaluate models on data cleaning, validation and
robustness tasks.

An optional partial log of injected defects is provided in
`intended_defects_manifest.json`. **Exclude that file if you want the model
to detect problems without labels.** Review and supplement it before using it as supervised
training or evaluation targets.

## Files

| File | Description |
|---|---|
| `plots.csv` | Plot-level records (~44 rows incl. duplicates) |
| `trees.csv` | Tree-level records (~500 rows incl. duplicates and broken rows) |
| `soil_profiles.json` | Nested soil horizon profiles for a subset of plots |
| `field_notes.txt` | Messy free-text crew notes |
| `species_reference.csv` | Clean species lookup table |
| `data_dictionary.csv` | Clean field dictionary |
| `intended_defects_manifest.json` | Partial log of injected defects (optional) |
| `loader_example.py` | Dependency-free example of defensive loading |
| `LICENSE.txt` | Complete MIT License |

## Defect categories present

* **Missing values** - blank, `NA`, `N/A`, `NaN`, `.`, `-999`, `-9999`,
  `null`, `None`, `not recorded`, whitespace-only.
* **Duplicates** - exact duplicate rows; near-duplicates sharing the same
  `plot_id` + `tree_number` key but with differing values; a duplicated
  plot record; a duplicated soil-profile object.
* **Corrupted numerics** - letters inside numbers (`1O.5`), embedded units
  (`14 (est)`), comma decimals (`12,5`), free text (`unknown`, `TBD`).
* **Out-of-range values** - `dbh_in` of 999 or -5, `slope_pct` of 240,
  canopy cover above 100%, negative elevation.
* **Unit mismatches** - heights recorded in metres inside `*_ft` columns.
* **Categorical inconsistency** - case changes, doubled internal spaces,
  non-breaking spaces, accented characters, ` spp.` suffixes, hyphen variants
  in `forest_type`.
* **Code/name mismatch** - `spcd` disagreeing with `species_common`.
* **Date problems** - six incompatible formats, plus impossible dates such
  as `2019/13/45` and `31-Feb-2020`.
* **Coordinate errors** - latitude and longitude swapped.
* **Structural breakage** - repeated header rows mid-file, blank lines,
  rows with an extra field (column shift).
* **Nested JSON problems** - missing required key, wrong value type,
  `"NA"` inside a numeric field, truncated horizon list.

## Suggested tasks

1. Cleaning pipeline: normalise dates, coerce numerics, canonicalise species.
2. Duplicate detection: exact vs. near-duplicate resolution.
3. Validation rule authoring (range, referential integrity, code lists).
4. Anomaly detection / outlier flagging.
5. Text-to-SQL or table-QA over inconsistent schema.
6. Imputation of missing `dbh_in`, `total_height_ft`, `crown_ratio_pct`.
7. Provenance reasoning: reconstruct which rows are trustworthy.

## Scientific notes

Values were drawn from plausible ranges for the region: mixed spruce-fir,
northern hardwood, mixedwood, white pine-hemlock, aspen-birch, cedar-swamp
and oak-pine communities; elevations roughly 0-2,400 ft; site index 38-92 ft
base age 50; DBH 1-34 in; heights derived from a power function of DBH
modulated by site index. Per-tree basal area uses 0.005454 x DBH^2.
Species codes are **plausible FIA-style codes but are not authoritative** -
verify against the official FIA species list before any real-world use.

## License

The generator, synthetic dataset and accompanying documentation are released
under the MIT License. Copyright (c) 2026 Hayven Jackson / Krummholz AI.
See LICENSE.txt for the complete terms.