Data Clean Env
An OpenEnv-compliant reinforcement-learning environment for tabular data cleaning. An agent fixes messy datasets through a typed action space (convert types, fill missing, dedupe, standardize, parse dates) and is scored per issue resolved. Built for the Meta OpenEnv Hackathon.
OVERVIEW
An OpenEnv-compliant reinforcement-learning environment for tabular data cleaning, built for the Meta OpenEnv Hackathon. An agent fixes messy datasets through a closed, typed action space of seven operations (replace value, convert type, fill missing, remove duplicates, drop rows, standardize, parse date) and is scored on the fraction of each task's checks that pass. The environment subclasses OpenEnv's Action and Observation types, exposes reset/step over a FastAPI server, and ships in Docker to run as a Space, with four graded tasks (from a 5-row type-fixing warm-up to a 15-row full pipeline), a reference client and inference format, and tests for the client, graders, and inference.
ARRIVED AS
Data cleaning is the unglamorous core of data engineering, fixing types, nulls, duplicates, and inconsistent formatting, and it is exactly the kind of structured, checkable task an agent could learn. The problem was framing it as a reinforcement-learning environment: a clean action space, observations an agent can reason over, and a reward that rewards fixing real issues rather than just changing cells.
Built for the Meta OpenEnv Hackathon, this is a reinforcement-learning environment, not a trained agent, whose job is to make tabular data cleaning learnable and scorable. Real datasets arrive with type errors, missing values, duplicates, and inconsistent formatting; a data engineer fixes them with a small repertoire of operations. The environment encodes that repertoire as a typed action space, presents the messy data and its current quality as observations, and scores progress against a fixed set of checks, so an agent can be trained or evaluated on a task that mirrors actual data work.
WHAT I BUILT
- 01An OpenEnv-compliant RL environment (built on the OpenEnv Action/Observation base types) so it plugs into the standard tooling rather than inventing its own protocol.
- 02A typed action space of seven cleaning operations: replace_value, convert_type, fill_missing, remove_duplicates, drop_rows, standardize (case/trim), and parse_date.
- 03Rich observations: the current data, per-column metadata (dtype, null counts, unique counts), a running quality score in [0, 1], and issues-fixed-versus-total progress, so the agent can see what is still wrong.
- 04Four graded tasks, from a 5-row type-fixing warm-up to a 15-row full-pipeline mix of every issue, served by a FastAPI app and packaged in Docker for deployment as a Space.
WHAT CHANGED
- Turns a fuzzy chore into a checkable RL task: each fixed issue is worth a fraction of the score, so the reward tracks real data-quality improvement rather than activity.
- OpenEnv compliance means the environment is interoperable with the standard client and inference tooling instead of being a one-off harness.
- A reference client, inference format, and grader tests ship alongside, so the environment is runnable and verifiable, not just a description.
Data flow
click a stage
The environment loads one of four tasks: a dirty dataset plus the checks that define a clean result, and returns the initial observation.
COMPONENT
tasks/task_data.pyThe four graded tasks: dirty datasets paired with the checks that define a clean result, from fix_types to full_pipeline.
Decisions, with the cost of each.
A decision without its trade-off is marketing. Each row says what was chosen, why, and what it gave up.
Build to the OpenEnv spec instead of a custom harness
A one-off environment only works with its own glue code. Subclassing OpenEnv's Action and Observation types makes the environment interoperable with the standard client and inference tooling, which is the whole value of a shared environment standard.
A bespoke gym-style API (works in isolation, but not with the OpenEnv ecosystem the hackathon was about).
A small typed action space, not free-form edits
Letting an agent write arbitrary transformations is hard to score and easy to game. Seven explicit operations (convert, fill, dedupe, standardize, parse_date, and so on) keep actions checkable and map onto what a data engineer actually does.
Free-form code or SQL actions (powerful but unsafe and hard to reward); a single 'clean' action (no learning signal).
Score per check passed, not per action taken
Rewarding actions would reward busywork. Tying the score to the fraction of a task's checks that pass means the reward measures actual data-quality improvement, and an agent that thrashes without fixing issues gains nothing.
Reward per action (incentivizes noise); all-or-nothing reward at the end (sparse, hard to learn from).
The part that mattered.
The numbers behind the work, and the code that produced them.
- graded scenarios
- 4 tasks
- fix_types → full_pipeline, 5-15 rows
- typed action space
- 7 actions
- convert · fill · dedupe · standardize · parse_date · ...
- spec-compliant
- OpenEnv
- Meta OpenEnv Hackathon
- per-check reward
- score 0-1
- issues_fixed / total_issues
from openenv.core.env_server.types import Action, Observation
class DataCleanAction(Action):
action_type: Literal[
"replace_value", "convert_type", "fill_missing",
"remove_duplicates", "drop_rows", "standardize", "parse_date",
]
column: Optional[str] = Field(None, ...)
target_type: Optional[Literal["int", "float", "str"]] = Field(None, ...)
method: Optional[Literal["lowercase", "trim", "title_case", "uppercase"]] = ...
# plus value / old_value / new_value / row_indices / subset_columns
class DataCleanObservation(Observation):
current_data: Optional[List[Dict[str, Any]]] = Field(None, ...)
column_info: Optional[Dict[str, Dict[str, Any]]] = Field(None, ...)
total_issues: int = 0
issues_fixed: int = 0
score: float = Field(0.0, description="Current score [0.0-1.0]")
done: bool = False
Action and observation subclass OpenEnv's base types, so the environment speaks the standard protocol. The action space is a closed set of seven typed operations, and the observation gives an agent everything it needs: the data, per-column metadata, progress, and the current score.
def _apply_action(data, action: DataCleanAction):
"""Apply an action to the data. Returns (new_data, result_message)."""
if action.action_type == "convert_type":
for row in data:
val = row.get(action.column)
if val is None:
continue
if action.target_type == "int":
row[action.column] = int(float(str(val)))
elif action.target_type == "float":
row[action.column] = float(str(val))
elif action.target_type == "str":
row[action.column] = str(val)
return data, f"Converted values in '{action.column}' to {action.target_type}"
# ... one branch per action type, each returning a result message
Each step dispatches on action_type to a handler that transforms the data and returns a human-readable result. After the transform the environment recomputes per-column metadata and the quality score, so the next observation reflects exactly what changed.
✓ LEARNED
Framing data cleaning as a closed, typed action space is what makes it learnable: open-ended edits are powerful but neither safe nor scorable.
Rewarding checks passed rather than actions taken keeps the signal honest, an agent only scores by actually improving data quality.
Building to a shared spec like OpenEnv costs a little structure up front and buys interoperability with the standard tooling, which is the entire point of a common environment interface.