Reinforcement learning for environmental work: ecbyts as the training environment
About 7 min read
- Guide
- Audio
- Tour
Training an algorithm to make decisions about a contaminated site runs into a practical problem: there is nowhere to be wrong without creating liability. ecbyts steps in as the environment — the digital twin that returns state, reward and consequence.
A data science team can train an agent to play chess in an afternoon. Ask the same team to decide where to drill an investigation well on a contaminated site — a GAC project, in Brazilian practice — and the project stops. Not for lack of algorithms: because there is no safe place for the agent to be wrong.
Errors in the field are expensive and slow to surface: a sampling campaign takes weeks from mobilization to validation, and a misplaced well does not count as a failed attempt — it counts as liability.
Before choosing the algorithm, the question that decides the project is another one: where can this agent be wrong a thousand times without creating liability?
What reinforcement learning is, without the jargon
Reinforcement learning teaches a program by trial and consequence, like training a dog: right answer, a point; wrong answer, nothing. The difference is that the point is arithmetic, not a biscuit.
Five words carry everything:
- Agent: the program that decides.
- State: the snapshot of the world at that moment — what was measured, where, when, and what is still missing.
- Action: what it chooses to do: commission a campaign, change the remediation route, prioritize a point.
- Reward: the number that says whether the action was good.
- Policy: the rule that maps state to action, and the thing you take to the field.
A full round — see, act, collect the reward, correct the policy — is an episode. A lab episode costs milliseconds; a field one costs money and schedule.
The bottleneck is the environment, not the algorithm
There are ready-made libraries for almost every reinforcement learning algorithm. What does not exist is an environment that answers "what happens if I do this?" with a defensible number.
To serve as an environment, a system must be fast, deterministic, grounded in real data, auditable, and honest about what it does not know. A spreadsheet fails all five: it does not propagate uncertainty, does not reject physically impossible combinations, and does not track which data version entered the math.
ecbyts as the environment: the digital twin that answers back
ecbyts is an environmental digital twin: the project model inside the browser, with elements (wells, areas, sources), observations dated by campaign and calculations that keep a trail. The state of the environment is that model, not a prose description of it.
For an external agent, the state arrives through three routes: the Agent Hub, which shows exactly what an agent reads from the model — elements, campaigns, gaps and tools; the API; and MCP, the open protocol that connects assistants to tools. Reads are deterministic and travel with proof: an integrity stamp (the proofStamp, a SHA-256 fingerprint) and a Merkle tree, which proves an item was not swapped without resending the whole set.
Two guards matter for anyone training:
- The hosted MCP is read-only today: it delivers state, it does not accept commands.
- Writes require two-sided approval — whoever runs the agent and whoever is accountable for the model must both agree.
And the limit that is not a promise: direct actuation on sensors and valves in real time is Vision — not available yet.
The shortest way to see this working is the guided tour of the reinforcement learning targets.
The targets: where the reward is calculated
ECBT Score: the balance, with uncertainty inside the number
ECBT Score compares the environmental value that exists today with the cost of resolving the liability and returns a balance in Brazilian reais, next to ECBT %, the ratio between the two sides. The large number is the balance; the band around it is the uncertainty.
The width of that band depends on data confidence, summarized by κ (the Greek letter kappa): the lower the κ, the wider the band, which can reach three times its base width. So a fragile gain cannot be read as a certain one. See it in the ECBT Score panel. And remember: it is a simulation indicator, not a technical report.
EIS: the evidence integrity score, from 0 to 5
EIS scores evidence from 0 to 5 across four fronts: coverage (how much was measured), method (whether collection followed a recognized procedure), audit (whether someone independent checked it), and trace (whether every number can be traced back).
It is what defines κ: κ = EIS ÷ 5. An EIS of 5 gives κ = 1, maximum confidence; an EIS of 2.5 gives κ = 0.5, and the ECBT Score band opens. The EIS panel shows the score ring and each criterion's contribution.
The Calculator as an optional target
When the goal is neither money nor an overall score, the Calculator builds the target to measure: the average of a parameter, a ratio between two parameters, or a rule against a legal limit — for instance, how many wells came out above the investigation value. Ready-made items such as ECBT Score appear in the same list.
Use the rule when the goal is a hard limit; use a ratio when proportion, not absolute value, counts.
The action, tested first: the scenario comparator
An agent needs actions with visible effect. In the scenario comparator, the Base stays fixed, each scenario changes one hypothesis, and the app rejects physically impossible combinations, such as recovery higher than extraction. The effect shows up in ECBT Score and in the SDGs. And the warning still holds: simulation is not measurement.
Why ECBT Score and EIS together close the shortcut
If the reward is money alone, the shortest path is to cut cost — and the easiest cost to cut in an environmental project is monitoring. The agent "learns" to run fewer campaigns, the balance improves on paper, and the liability stays right where it was, now with less evidence on top. That is the single-axis reward problem: the algorithm optimizes the number you gave it, not the intent you had.
Two things break the shortcut:
- EIS is part of the reward. Cutting campaigns lowers coverage, and the score drops.
- κ drops with it. The ECBT Score band opens, and the gain stops being certain.
What remains is a two-axis reward: balance and confidence. The agent only wins when it wins on both. Whoever writes the target writes the behavior.
Training a fast surrogate model inside the app
Running the full engine thousands of times is expensive. The way out is a surrogate model: a small network that imitates the engine's output with a known error, sweeps the space of combinations in seconds, and leaves the interesting points for confirmation in the full engine.
ecbyts does this in ML Studio, in the browser, with no server or Python. You pick inputs and output; calculated variables — EIS, ECBT Score and Calculator items — enter as input or output, so the indicator can be a training target.
Three cautions: the network estimates, it does not measure; outside the range of the training data the estimate is a guess with the appearance of a number; and a surrogate speeds up the search but does not decide. No neural network and no learned policy changes measured data or signs in place of the technical responsible — the engineer who holds the ART. The ECBYTS Seal exists to separate, on screen, what is machine from what carries a human ART and attestation.
Honest limits
- Simulation is not measurement. Where there is no measurement, there is a declared estimate.
- Demo data is fictional. The numbers in the tour describe no real site.
- Sensors and valves in real time: Vision — not available yet.
- Hosted MCP: read-only; writes go through two-sided approval.
- An indicator is not a report. It does not replace a technical report, a regulator's opinion or an ART.
- The environment carries the limits of the data it received. A missing campaign is a visible gap, never a zero.
Conclusion
The bottleneck for reinforcement learning in environmental work was never the math. It was the absence of a place where being wrong is cheap and where consequence shows up as a comparable number. ecbyts provides that place: state an agent can read, actions tested in scenarios, and a reward built on two indicators that keep each other honest.
You choose the algorithm. You answer for the reward.
Start with the guided tour of the targets and take one question into the first scenario: which shortcut will your agent find before it finds the solution?